repr(ordered_fields)

[RustyYato]

Add repr(ordered_fields) and provide a migration path to switch users from repr(C) to repr(ordered_fields), then change the meaning of repr(C) in the next edition.

This RFC is meant to be an MVP, and any extensions (for example, adding more reprs) are not in scope. This is done to make it as easy as possible to accept this RFC and make progress on the issue of repr(C) serving two opposing roles.

Rendered

To avoid endless bikeshedding, I'll make a poll if this RFC is accepted with all the potential names for the new repr. If you have a new name, I'll add it to the list of names in the unresolved questions section, and will include it in the poll.

[clarfonthey]

Not to add too many extra colours to the list, but repr(consistent) feels like a good name for this, since the purpose is to provide a consistent layout that does not depend on generics, compiler version, or target. The important thing is just that it's consistent, not that it matches what C does.

(Note: those three things should cover every case I've seen that uses repr(C) that should use repr(ordered_fields), but please feel free to correct me if I missed anything.)

Whereas repr(C) is explicitly, match what C does.

Also, while it may be more technical than most users need to understand, it would be helpful if the RFC reiterated the current issues with repr(C) that we want to fix, and potential future differences between repr(C) and repr(ordered_fields) that could pop up. I've read some of them but am not 100% sure of the details, and it would be nice to keep as part of the RFC.

[Lokathor]

Just as a small point of style the Guide Level Explanation is usually "what would be written in the rust tutorial book", and the Reference Level Explanation is "what could be written into the Rust Reference". This isn't a strict requirement, but personally I'd like to see the Reference Level part written out. Using the present tense, as if the RFC was accepted and implemented.

[programmerjake]

Suggested change

	`repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in memory layout.
	`repr(ordered_fields)` is a new representation that can be applied to `struct`, `enum`, and `union` to give them a consistent, cross-platform, and predictable in-memory layout.

"cross-platform" -- the layout will differ when there are different layouts for struct members' types, in particular primitive types can have different alignments which changes the amount of padding.

e.g., #[repr(ordered_fields)] struct S(u8, f64); doesn't have the same layout on x86_64 and i686

[RustyYato]

Good point, this will need to be documented as a hazard in the ordered_fields docs. However, the repr itself will be cross-platform. For example, #[repr(ordered_fields)] struct Cross([u8; 3], SomeEnum); will be truly cross-platform (given that SomeEnum is!).

[moonheart08]

Not to add too many extra colours to the list, but repr(consistent) feels like a good name for this, since the purpose is to provide a consistent layout that does not depend on generics, compiler version, or target. The important thing is just that it's consistent, not that it matches what C does.

(Note: those three things should cover every case I've seen that uses repr(C) that should use repr(ordered_fields), but please feel free to correct me if I missed anything.)

Whereas repr(C) is explicitly, match what C does.

Also, while it may be more technical than most users need to understand, it would be helpful if the RFC reiterated the current issues with repr(C) that we want to fix, and potential future differences between repr(C) and repr(ordered_fields) that could pop up. I've read some of them but am not 100% sure of the details, and it would be nice to keep as part of the RFC.

Just voicing support for repr(consistent) as naming.
Aside from the above, it more clearly hones in on the primary promises of the RFC, which is not just ordering but also exact type representation for things like enums. Field ordering is not the only thing it promises.

[joshtriplett]

Nominating this so that we can do a preliminary vibe-check on it in a lang triage meeting.

[scottmcm]

Big fan of doing this split, especially for structs. (It's less obvious what choices to make for other things, IMHO, but at least for structs this is something I've wanted for ages, so that for example Layout::extend can talk about it instead of C.)

Pondering the bikeshed: declaration_order or something could also be used to directly say what you're getting.

(This could be contrasted with other potential reprs that I wouldn't expect this RFC to add, but could consider as future work, like a deterministic_by_size_and_alignment where some restricted set of optimizations are allowed but you can be sure that usize and NonNull<String> can be mixed between different types while still getting the "same" field offsets, for example.)

[RustyYato]

I think this is also useful for unions, so we don't need to rely on repr(C) to ensure that all fields of a union are at offset 0.

This could be contrasted with other potential reprs that I wouldn't expect this RFC to add...

This also works as an argument against names like repr(consistent), since there are multiple consistent and useful repr, making it not descriptive enough.

[scottmcm]

I do think that declaration_order or ordered_fields is a bit weird on a union, because of course they're not really in any "order".

It makes me ponder whether we should just have repr(offset_zero) for unions to be explicit about it, or something.

(Which makes me think of other things like addressing rust-lang/unsafe-code-guidelines#494 by having a different constructs for "bag of maybeuninit stuff that overlap" vs "distinct options with an active-variant rule for enum-but-without-stored-discriminant". But those are definitely not this RFC.)

[RustyYato]

I don't mind spelling it as repr(offset_zero) for unions if that helps get this RFC accepted 😄. However, I have a sneaking suspicion that this isn't the contentious part of this RFC.
I know the name isn't optimal (intentionally). This can be hashed out after the RFC is accepted (or even give a different name for all of struct, union, and enum).
The most important bit for me is just that we do the split (for all of struct, union, and enum, to be consistent).

[RustyYato]

I've updated how enums's tags are specified, now they just defer to whatever repr(C)'s tag type is. This is done to reduce the friction of switching from repr(C) to repr(ordered_fields). To ensure that all uses of repr(ordered_fields) can be cross-platform, I've adding a lint to ensure that the user also adds an explicit repr for repr(uN)/repr(iN).

[Mark-Simulacrum]

I think this is too noisy. Most code out there using repr(C) is probably fine - IIUC, if you're not targeting Windows or AIX, maybe definitely fine? - and having a bunch of allow(...) across a bunch of projects seems unfortunate.

Maybe we can either (a) only enable the lint for migration, i.e., the next edition's cargo fix would add allows for you or (b) we find some new name... C2 for the existing repr(C) usage to avoid allows. But (b) also seems too noisy to me.

[clarfonthey]

Maybe it could just be an optional edition compatibility lint, so if someone enables e.g. rust_20xx_compatibility it shows up but otherwise not.

[RustyYato]

(a) only enable the lint for migration

That was the intention, hence the name edition_2024_repr_c. I'll make this more clear, that this is intended to be a migration lint.

Rustfix would update to #[repr(ordered_fields)] to preserve the current behavior. For the FFI crates, #![allow(edition_2024_repr_c)] at the top of lib.rs would suffice. If you have a mix of FFI and non-FFI uses of repr(C), then you'll have to do the work to figure out which is which, no matter what option is chosen to update repr(C) - even adding repr(C2), since then the FFI use case would need to update all their reprs to repr(C2).

Overall, I think this scheme only significantly burdens those who have a mix of FFI and non-FFI uses of repr(C). But they were going to be burdened no matter what option was chosen.

[RustyYato]

Is the new wording/lints too noisy still?

[joshtriplett]

I think the new wording is still too noisy. We shouldn't assume that most people using repr(C) are using it for ordering rather than FFI.

[RustyYato]

That wasn't my intention, but I don't see another way to do all of the following in the next edition:

• make repr(C) mean - same layout/ABI as what the standard C compiler does
• make repr(ordered_fields) - the same algorithm that's listed for repr(C) in the Rust reference
• ensure that everyone who upgrades to the next edition gets the layout they need (as long as they read the warnings and follow the given advice)
• make it as painless as possible for people who don't mix FFI and stable ordering cases (which I suspect is the vast majority of people). In other words, each crate currently uses repr(C) either exclusively for FFI or exclusively for some stable layout.
• for people who do mix FFI and stable ordering cases in one crate, at least the warning should give them all the places they need to double-check, and they can silence the warning on a case-by-case basis.

I'm open to suggestions on how to handle the diagnostics. Within these constraints, I think my solution is the only real option we have. If there are some objections to these constraints, I would like to hear those too, maybe I missed the mark with these constraints, and missed a potential solution because of it.

[programmerjake]

maybe there should be a warning for crates that use repr(C) in >edition 2024 but don't have any extern "<non-Rust>" blocks or functions, since that's a good indicator they're probably not using it for FFI and haven't heard about repr(ordered_fields)

[RustyYato]

That's a good idea!

[RustyYato]

I think any extern block/function should disable the lint, even extern "Rust" to allow Rust-Rust FFI (with some stable ABI, i.e. a future crABI or the abi_stable crate).

[programmerjake]

if you're doing Rust-Rust FFI and expect any kind of ABI stability, you shouldn't be using extern "Rust" (since that's explicitly not stable), you should be using extern "C" (used by abi_stable afaict) or extern "crabi", both of which would disable the lint according to my proposal.

[Jules-Bertholet]

Two repr(Rust) structs with the same field names, types, and order are not guaranteed to have the same layout and ABI. But two extern "Rust" functions with the same signature, on the same platform and compiler version are guaranteed ABI-compatible. So there is a use-case for combining repr(C) with extern "Rust".

[RustyYato]

I've added the new lint to all editions, and it should take precedence over the migration lint. This way, we can migrate as many people as possible to repr(ordered_fields). Making it much easier to read code in the future, at the cost of some churn now.

[kpreid]

This statement matches the algorithm currently used for "primitive representations" (repr(u8) etc), not the algorithm currently used for repr(C) or repr(C, uN). If this change is made, then anyone in need of specifically the current repr(C) enum representation cannot migrate to repr(ordered_fields).

The difference between the two algorithms (besides the integer type used for the tag) is that in repr(C), all variants have their first field at the same offset, whereas in repr(uN), each variant’s first field has only the offset required by its type’s alignment, so individual variants may be at a lower offset. The repr(C) enum acts as (and is documented as) a “tagged union” that contains an “untagged union”, whereas repr(uN) does not.

Example of the difference:

#![feature(offset_of_enum)]

#[repr(align(8))]
struct A64(u64);

#[repr(C, i32)]
enum ExampleC {
    Foo(u8),
    Bar(A64),
}
#[repr(i32)]
enum ExamplePrim {
    Foo(u8),
    Bar(A64),
}

fn main() {
    dbg!(
        std::mem::offset_of!(ExampleC, Foo.0), // prints 8
        std::mem::offset_of!(ExamplePrim, Foo.0), // prints 4
    );
}

I think that either repr(ordered_fields, uN) should instead exactly mimic repr(C, uN) in this regard, or — perhaps a better idea — there should be a separate explicitly named representation for this, since it is rare to actually need this layout, and so users merely seeking predictable layout are better served by repr(uN), but they might unthinkingly put repr(ordered_fields) on their enums along with their structs..

[RustyYato]

That's a good point. I didn't realize there was that difference, thanks for bringing this up. I think repr(ordered_fields) should match what repr(C) does now, to ease migrations (even in combo with the primitive reprs). We could add lints to suggest changing to just a primitive repr if repr(ordered_fields) is used on an enum.

I think making the migration as painless as possible will make it much easier to split up repr(C) up. And we can also add guidance towards other reprs via lints.

I'm open to having the repr be named differently for the different kinds of type. (For example, repr(offset_zero) for unions was considered in another thread)

[RalfJung]

If that's the third motivation, then what is the secondary motivation?^^

[RustyYato]

I guess it wasn't super clear,

The motivations are

1. Splitting up two contradictory use-cases for repr(C)
2. Correct handling of ZSTs on MSVC repr(C) on MSVC targets does not always match MSVC type layout when ZST are involved rust#81996
3. The MSVC alignment bug MSVC on x86-32 Windows fails to align variables to their required alignment rust#112480

But also, I was trying to hint that this isn't the most important motivation. I'll reword this to be less confusing.

[RalfJung]

I don't see how 1 is a motivation in itself -- 2 and 3 are the motivation to do 1. If it wasn't for 2 and 3, why would we care about 1? I don't think the idealistic "these two cases should not be conflated" really stands as a motivation (for a breaking change!) if there's no downside to conflating them.

[RustyYato]

Ok, I've updated the motivation section to remove references to numbers, so the motivation section should flow better.

I don't think the idealistic "these two cases should not be conflated" really stands as a motivation (for a breaking change!) if there's no downside to conflating them.

Yes, I agree that this doesn't stand on it's own. However, it doesn't need to, since it is one (small) motivation among others that make the case for this change. A small motivation to help add context to other more important motivations.

In this case, there is a downside to conflating them (as seen in the other linked issues). I guess I could argue that this is really the only motivation, with the others as evidence that there are downsides to this conflation. But that feels too much like arguing about semantics, which I would like to avoid.

[jackh726]

The RFC does a good job at establishing that there are two potentially distinct use for today's repr(C). What the RFC doesn't do (and I think is important) is evaluate why it's actually a problem to conflate these two. Specifically curious what use cases break if we just "fix" reprC).

[Jules-Bertholet]

I think would be nice to guarantee that if a repr(ordered_fields) ADT is layout-compatible (same size, alignment, and field offsets, discounting repr(Rust) ZST fields) with a struct or union defined in C, then it is also ABI-compatible with that C struct/union, as far as possible. This would allow using repr(ordered_fields) to interoperate with C code that defines types with weird pragmas, like AIX #pragma align (natural).