Add @backingInt and @fromBackingInt for converting between packed structs, packed unions, enums and their integer representations #35602

Migrated from: https://github.com/ziglang/zig/issues/18882

Accepted proposal (as per this comment):

• Remove @intFromEnum, replacing it with a new builtin @backingInt. That builtin accepts enums and bitpacks (packed struct/packed union).
• Remove @enumFromInt, replacing it with a new builtin @fromBackingInt. That builtin can return enums and bitpacks (packed struct/packed union).
• Allow @bitCast to accept and/or return enum types. This accepts the sub-proposal given in this comment.

This also effectively accepts https://github.com/ziglang/zig/issues/22712, because @fromBackingInt will provide a result type to its operand.

I've migrated this issue because for me there are a few open questions remaining:

• This regresses using @intFromEnum directly on tagged unions, as @backingInt only works on packed unions. Is that intentional?

• What does 'removing all restrictions' from @bitCast mean? Should it check whether a tag value is valid when casting to an exhaustive enum or just not perform any safety checks at all? What about pointers and error sets?

• What about enums/bitpacks with inferred backing integers? Should @backingInt/@fromBackingInt work on those or should one use @bitCast with an explicit result type instead?

• Bonus question: What about invalid enum tag values? If @bitCast doesn't perform any safety checks they can now be constructed without much friction, and as #35486 and #35448 show they can already be constructed easily as part of bitpacks today. How should they be handled?

A lot of the below isn't necessarily a consensus within the core team (we've not discussed it), but rather just my own opinions. I think that an initial implementation of this which works the way I've written here would likely be accepted.

This regresses using @intFromEnum directly on tagged unions, as @backingInt only works on packed unions. Is that intentional?

Unintentional, sorry for not catching that! @backingInt should also work on tagged unions by just using the tag.

What does 'removing all restrictions' from @bitCast mean?

Hm, did I use this wording somewhere? I can't immediately see it.

Should it check whether a tag value is valid when casting to an exhaustive enum [...] ?

I think from a language perspective it might make the most sense to not do this safety check and to instead return undefined if the tag is not valid. However, since we don't currently have runtime safety for undefined (tracked by #63), that would probably be a major footgun right now. Therefore, for now, let's say that there is a safety check for this (the same one as @fromBackingInt).

What about pointers...

I assume this is referring to pointer alignment checks? I would say the same as above---i.e. for now let's include the safety-check, but in the future we might want to revisit that decision.

...and error sets?

Error sets are not and should not be accepted by @bitCast (neither as the operand type nor the result type). The bit-level representation and byte-level ("in memory") representation of errors are both considered to be opaque and implementation-defined, so the only valid way to convert them to/from the underlying error code integer is @errorFromInt/@intFromError.

What about enums/bitpacks with inferred backing integers?

I would suggest not allowing @backingInt/@fromBackingInt on those. With this, we come closer to removing the concept of "inferred backing types" from Zig, which I think is a good idea and meshes super nicely with #24714---in such a world, given a packed struct { x: u8 }, we can say that the type is 8 bits long, but we cannot say that it is backed by specifically a u8. It's a good idea to be initially conservative with what we allow, because we can relax a restriction in future without breaking anyone's code, but adding a restriction is necessarily breaking. In this case, I would definitely argue that it makes sense to require people to use @bitCast for these types so that they are forced to acknowledge the type they are using!

Bonus question: What about invalid enum tag values?

Firstly, I should note that I think I've said contradictory things about this before, so sorry for any confusion. I'm not 100% sure where I think the language should land here, but I would say that for now, as I mentioned above, @bitCast should have a safety check for this.

As you correctly note, the language problem still very much exists, per #35486 and #35448. The solution to this is not obvious, so definitely don't try to deal with it while implementing this proposal! At some point I'll try to speak to the rest of the core team about this design issue so that we can put together a proposal for the semantics here.

What does 'removing all restrictions' from @bitCast mean?

Hm, did I use this wording somewhere? I can't immediately see it.

Sorry the exact wording was from memory didn't double check, I'm referring to this comment by Andrew

What about pointers...

I assume this is referring to pointer alignment checks? I would say the same as above---i.e. for now let's include the safety-check, but in the future we might want to revisit that decision.

I was referring to pointers working at all with @bitCast, I'm guessing the answer is 'yes'?

If yes, what's the plan for @intFromPtr and @ptrFromInt? Should @bitCast on pointers just be the same but without the usize result type?

Oh wait, sorry, brain was clearly slightly switched off when I wrote my comment; pointers should not be legal as @bitCast operand or destination type! (This actually was agreed internally---one reason for this is because avoiding treating pointers as simple sequences of bits makes for a better provenance model and may make it easier to support CHERI architectures in the future.)