dnkl commented 2022年01月02日 20:14:56 +01:00

Summary

foot now has its entire terminfo builtin, and will respond to XTGETTCAP queries for all capabilities, including the XTGETTCAP specific TN, Co and RGB ones.

For reference, XTerm only sends replies for keyboard escapes (i.e. k* capabilities). Kitty sends replies for everything (as far as I can tell), except for boolean capabilities.

Details

We emit one DCS reply for each queried capability (like Kitty, but unlike XTerm), as this allows us to a) not skip any capabilities in the query, and b) reply with correct success/fail flag for each capability.

Similar to Kitty (but again, unlike XTerm), we support quering all terminfo capabilities. In unparameterized capabilities, \E is "expanded" to a literal ESC, while in parameterized capabilities, \E is kept as-is (this is identical to Kitty's behavior).

We do not batch the entire reply - as soon as the reply for one capability is done, we write it to the PTY.

(削除) In this iteration, boolean capabilities are not supported. Not even Kitty supports this. (削除ここまで)

We now reply with DCS 1 + r <capability> ST for boolean capabilities. That is, a normal "success" reply, minus the = <value> part. Note that foot is the only terminal emulator I'm aware of that does this. Kitty does not respond with any boolean capabilities.

Closes #846

https://codeberg.org/attachments/f4834e44-9967-4816-96e8-20c473630504
Image shows a mix of boolean, integer and string capabilities being queried, as well as one invalid capability.

Size

~(削除) This adds 20K to the final binary size. In the variant currently used here (const char * for each capability/value pair), each capability uses 16 bytes (two 8-byte pointers), plus the size of the actual strings. However, this results in lots of relocations, which increases the binary size further. (削除ここまで)

~(削除) If we instead use a table with fixed-sized char arrays (allowing for the longest capability names/values), then we lose the relocations, but the table itself becomes 20K. (削除ここまで)

The internal representation has been changed, from a table, to a single NULL-separated char array. This drops all extra pointers, and all extra relocations. The array itself is ~3.3K - much better than the 20K we saw before.

# Summary foot now has its entire terminfo builtin, and will respond to `XTGETTCAP` queries for **all** capabilities, including the `XTGETTCAP` specific `TN`, `Co` and `RGB` ones. For reference, XTerm only sends replies for keyboard escapes (i.e. `k*` capabilities). Kitty sends replies for everything (as far as I can tell), except for boolean capabilities. # Details We emit one DCS reply for each queried capability (like Kitty, but unlike XTerm), as this allows us to **a)** not skip any capabilities in the query, and **b)** reply with correct success/fail flag for each capability. Similar to Kitty (but again, unlike XTerm), we support quering **all** terminfo capabilities. In unparameterized capabilities, `\E` is "expanded" to a literal ESC, while in parameterized capabilities, `\E` is kept as-is (this is identical to Kitty's behavior). We do not batch the entire reply - as soon as the reply for _one_ capability is done, we write it to the PTY. ~~In this iteration, boolean capabilities are not supported. Not even Kitty supports this.~~ We now reply with `DCS 1 + r <capability> ST` for boolean capabilities. That is, a normal "success" reply, minus the `= <value>` part. Note that foot is the only terminal emulator I'm aware of that does this. Kitty does not respond with any boolean capabilities. Closes #846  Image shows a mix of boolean, integer and string capabilities being queried, as well as one invalid capability. # Size ~~This adds ~20K to the final binary size. In the variant currently used here (`const char *` for each capability/value pair), each capability uses 16 bytes (two 8-byte pointers), plus the size of the actual strings. However, this results in lots of relocations, which increases the binary size further.~~ ~~If we instead use a table with fixed-sized char arrays (allowing for the longest capability names/values), then we lose the relocations, but the table itself becomes ~20K.~~ The internal representation has been changed, from a table, to a single NULL-separated char array. This drops all extra pointers, and all extra relocations. The array itself is ~3.3K - much better than the 20K we saw before.

dnkl commented 2022年01月09日 16:24:57 +01:00

@dankamongmen I know you already use XTGETTCAP with the TN and RGB capabilities (which foot currently does not implement at all), but was wondering if the extended XTGETTCAP functionality implemented in this PR would be of interrest to you?

I find XTerm's XTGETTCAP is of limited use, but I also see no point in implementing anything more advanced, if no one's going to use it...

@dankamongmen I know you already use `XTGETTCAP` with the `TN` and `RGB` capabilities (which foot currently does not implement at all), but was wondering if the extended `XTGETTCAP` functionality implemented in this PR would be of interrest to you? I find XTerm's `XTGETTCAP` is of limited use, but I also see no point in implementing anything more advanced, if no one's going to use it...

dankamongmen commented 2022年01月10日 08:04:46 +01:00

i have https://github.com/dankamongmen/notcurses/issues/2144 open. ideally, i'd stop using terminfo and get all properties from the terminal itself. i don't see myself moving to mass XTGETTCAP anytime soon, due to a noted miserly nature regarding bytes, but the first person who implements a more reasonable XTGETTCAP (i'd like to see the ability to query multiple entries or all entries, and a less grotesque encoding) will see me adopt it immediately, and also know they will no longer have TERM/terminfo-related issues with Notcurses.

i have https://github.com/dankamongmen/notcurses/issues/2144 open. ideally, i'd stop using terminfo and get all properties from the terminal itself. i don't see myself moving to mass XTGETTCAP anytime soon, due to a noted miserly nature regarding bytes, but the first person who implements a more reasonable XTGETTCAP (i'd like to see the ability to query multiple entries or all entries, and a less grotesque encoding) will see me adopt it immediately, and also know they will no longer have TERM/terminfo-related issues with Notcurses.

dankamongmen commented 2022年01月10日 08:07:44 +01:00

oh hey, it looks like you can already query multiple entities in a single XTGETTCAP, and indeed that i am doing this:

// XTGETTCAP['TN', 'RGB'] (Terminal Name, RGB) 
#define XTGETTCAP "\x1bP+q544e;524742\x1b\\" 

oh hey, it looks like you can already query multiple entities in a single XTGETTCAP, and indeed that i am doing this: ```c // XTGETTCAP['TN', 'RGB'] (Terminal Name, RGB) #define XTGETTCAP "\x1bP+q544e;524742\x1b\\" ```

dnkl commented 2022年01月10日 08:36:26 +01:00

but the first person who implements a more reasonable XTGETTCAP (i'd like to see the ability to query multiple entries or all entries, and a less grotesque encoding) will see me adopt it immediately, and also know they will no longer have TERM/terminfo-related issues with Notcurses.

This PR does allow you to query all entries, at once. The query will of course be rather big, since there's no keyword for "give me everything".

And for compatibility reasons, I haven't really changed the encoding, besides adopting Kitty's behavior of sending one DCS response per capability.

But if we're going to be changing a lot, it's probably better to use a new sequence all together, rather than pretending to be implementing XTGETTCAP.

> but the first person who implements a more reasonable XTGETTCAP (i'd like to see the ability to query multiple entries or all entries, and a less grotesque encoding) will see me adopt it immediately, and also know they will no longer have TERM/terminfo-related issues with Notcurses. This PR does allow you to query all entries, at once. The query will of course be rather big, since there's no keyword for "give me everything". And for compatibility reasons, I haven't really changed the encoding, besides adopting Kitty's behavior of sending one DCS response per capability. But if we're going to be changing a lot, it's probably better to use a new sequence all together, rather than pretending to be implementing XTGETTCAP.

dankamongmen commented 2022年01月10日 08:40:05 +01:00

But if we're going to be changing a lot, it's probably better to use a new sequence all together, rather than pretending to be implementing XTGETTCAP.

agreed.

interestingly(?), as we've had this discussion, i've expanded my use of XTGETTCAP to include "hpa" because of some snafu regarding kitty's terminfo on freebsd that i don't care to investigate.

> But if we're going to be changing a lot, it's probably better to use a new sequence all together, rather than pretending to be implementing XTGETTCAP. agreed. interestingly(?), as we've had this discussion, i've expanded my use of XTGETTCAP to include "hpa" because of some snafu regarding kitty's terminfo on freebsd that i don't care to investigate.

dnkl commented 2022年01月10日 08:50:37 +01:00

I guess my question then is: is the current form of XTGETTCAP "good enough" for querying all capabilities?

My gut feeling is that the bulkiness of the protocol isn't going to be a problem (besides maybe being ugly...) since the terminfo is fairly small.

Or should we invest time in a new sequence?

I guess my question then is: is the current form of XTGETTCAP "good enough" for querying all capabilities? My gut feeling is that the bulkiness of the protocol isn't going to be a problem (besides maybe being ugly...) since the terminfo is fairly small. Or should we invest time in a new sequence?

dankamongmen commented 2022年01月10日 08:54:14 +01:00

i mean, i'm definitely being silly bitching about the encoding (though on even a 9600bps connection, it would be noticeable).

i mean, i'm definitely being silly bitching about the encoding (though on even a 9600bps connection, it would be noticeable).

dankamongmen commented 2022年01月10日 08:56:36 +01:00

XTerm only sends replies for keyboard escapes

is this true? maybe i have some configuration element set, but i'm pretty sure i get RGB from it; that was what motivated my initial use of XTGETTCAP, since i very much want 24-bit color if it's available.

> XTerm only sends replies for keyboard escapes is this true? maybe i have some configuration element set, but i'm pretty sure i get RGB from it; that was what motivated my initial use of XTGETTCAP, since i very much want 24-bit color if it's available.

dankamongmen commented 2022年01月10日 08:57:33 +01:00

XTerm only sends replies for keyboard escapes

is this true? maybe i have some configuration element set, but i'm pretty sure i get RGB from it; that was what motivated my initial use of XTGETTCAP, since i very much want 24-bit color if it's available.

from my XTerm:

tcap_cb:1486:xtgettcap [544e=787465726D2D323536636F6C6F72;524742=38]
tcap_cb:1523:got rgb (524742=38)

> > XTerm only sends replies for keyboard escapes > > is this true? maybe i have some configuration element set, but i'm pretty sure i get RGB from it; that was what motivated my initial use of XTGETTCAP, since i very much want 24-bit color if it's available. from my XTerm: tcap_cb:1486:xtgettcap [544e=787465726D2D323536636F6C6F72;524742=38] tcap_cb:1523:got rgb (524742=38)

dnkl commented 2022年01月10日 09:14:30 +01:00

Yes. XTGETTCAP defines three "special" capabilities: TN, RGB and Co.

Co is just an alias for colors.

RGB's semantic however, differs from a normal terminfo. In terminfo, RGB is a boolean, while in XTGETTCAP it's an integer that specifies the number of bits per channel (i.e 8).

Beyond those, XTerm only responds to key capability queries.

Yes. XTGETTCAP defines three "special" capabilities: TN, RGB and Co. Co is just an alias for `colors`. RGB's semantic however, differs from a normal terminfo. In terminfo, RGB is a boolean, while in XTGETTCAP it's an integer that specifies the number of bits per channel (i.e 8). Beyond those, XTerm only responds to key capability queries.

dnkl commented 2022年01月10日 20:33:50 +01:00

Alright, I just might proceed, and merge this. Just going to sleep on it. Maybe more than one night...

Alright, I just might proceed, and merge this. Just going to sleep on it. Maybe more than one night...

dnkl commented 2022年01月13日 13:45:55 +01:00

The internal representation has been changed, from a table, to a single NULL-separated char array. This drops all extra pointers, and all extra relocations. The array itself is ~3.3K - much better than the 20K we saw before.

The internal representation has been changed, from a table, to a single NULL-separated char array. This drops all extra pointers, and all extra relocations. The array itself is ~3.3K - much better than the 20K we saw before.