Why is the safety comment for this unsized type so confusing?

[joshlf]

This issue exists to document a pattern that crops up repeatedly in designs, and is confusing enough that it often requires explanation.

Consider this trait from the field projection design:

/// A type that supports field projection into `Self::Inner`.
///
/// Given `P: Projectable<F, W>`, if `P::Inner` has a field of type `F`, that field may be projected
/// into `W`, which is the wrapped equivalent of `F`.
pub unsafe trait Projectable<F: ?Sized, W: ?Sized> {
    type Inner: ?Sized;
}

We implement this trait for types like:

#[repr(transparent)]
pub struct Wrapper<T: ?Sized>(T);

unsafe impl<T: ?Sized, F: ?Sized> Projectable<F, Wrapper<F>> for Wrapper<T> {
    type Inner = T;
}

Naively, we might expect the safety comment on Projectable to read something like:

A type, P, may only be Projectable<F, W> if it is a repr(transparent), repr(C), or repr(packed) wrapper around another type, P::Inner. P may have other zero-sized fields, but may not have any other non-zero-sized fields. If a field, F, exists in P::Inner at byte offset f, then it must be sound to treat there as existing a type, W, at byte offset f in P.

However, this safety comment doesn't cover cases like the MaybeValid type introduced in the TryFromBytes design. That type is defined as (simplified for this explanation):

#[repr(transparent)]
pub struct MaybeValid<T: AsMaybeUninit + ?Sized>(T::MaybeUninit);

By design, T::MaybeUninit has the same layout as T, but MaybeValid is not literally a wrapper around T. Thus, we might instead write the safety comment on Projectable as:

A type, P, may only be Projectable<F, W> if it has the same size and field offsets as P::Inner. If a field, F, exists in P::Inner at byte offset f, then it must be sound to treat there as existing a type, W, at byte offset f in P.

However, this is problematic for unsized types, as we'll see in a moment.

An aside on unsized types

We need to support sized and unsized types. Specifically, we need to support the following types:

• Sized types
• Slice types ([T])
• Custom DSTs (types whose last field is a slice type)

We do not support dyn Trait types. Note that, in most cases, we can describe slice types as a degenerate type of custom DST - one in which the trailing slice field is the only non-zero-sized field in the type. This allows us to simplify some prose by not needing to describe slice types and custom DSTs separately.

While custom DSTs do not have a size which is known statically at compile time, each custom DST pointer or reference encodes the length of the trailing slice field. This is sufficient to determine the size of the referent of that pointer or reference. Thus, while we can't refer to an unsized type, T, as having a size, we can refer to a specific instance of T as having a size, and we can refer to a specific instance of &T or *const T as pointing to a T of known size.

Importantly, when converting between custom DSTs, raw pointer as casts preserve the number of elements in the trailing slice. In other words, given u: *const [u8], u as *const [u16] will result in a pointer to a slice of the same number of elements (and thus, in this case, of double the length). This is true for "real" custom DSTs (with leading sized fields) too.

Back to the main event

Recall our proposed safety conditions for Projectable:

A type, P, may only be Projectable<F, W> if it has the same size and field offsets as P::Inner. If a field, F, exists in P::Inner at byte offset f, then it must be sound to treat there as existing a type, W, at byte offset f in P.

This is problematic for unsized types, and it's unsized types that require us to make the safety comment significantly more convoluted. In particular, this safety comment doesn't support unsized types in the following ways:

• Unsized types don't have a fixed size, so it's nonsensical to refer to P and P::Inner as having the same size.
• Unsized types can have different field offsets depending on the instance of the type (e.g., a [u8] of length 3 has different field offsets than a [u8] of length 5), so it's nonsensical to refer to F existing at byte offset f in P::Inner or to W at byte offset f in P. Furthermore, the type F itself might be unsized, and so speaking only of its byte offset - rather than its byte offset and length - isn't sufficient to specify which range of bytes it lives in within P::Inner.

Given our aside on unsized types, we can see how to generalize the safety comment in order to address these shortcomings:

• Instead of referring to the sizes of P and P::Inner, we can refer to the size of a specific p: *const P. We need to ensure a few things:

  • P and P::Inner have to have the same sizedness - they must both be sized or must both be custom DSTs
  • If they're custom DSTs, their trailing slice elements must have the same size so that as casts preserve size; if this weren't the case, code that performed field projection would convert a *const P to a *const P::Inner and the latter pointer would have the wrong size

  In order to ensure both of these, we can simply say that:

  • It must be possible to perform let i = p as *const P::Inner; Rust ensures that this is only valid under the following circumstances, and so this rule disallows P sized while P::Inner is a custom DST
    • Converting from a sized type to a sized type
    • Converting from a custom DST to a custom DST
    • Converting from a custom DST to a sized type
  • p and i must point to objects of the same size; this both ensures all of the following:
    • If P and P::Inner are sized, they have the same size
    • If P and P::Inner are both custom DSTs, their trailing slice elements have the same size
    • If P is a custom DST while P::Inner is sized, then this condition cannot possibly hold for all p since p can have different sizes while P::Inner always has one size; thus, this condition is ruled out

• Instead of referring to a field of type F as existing at offset f of P::Inner, we can refer to a field of type F existing at byte range f within an instance i: &P::Inner

Putting all of the pieces together, we get the following safety condition for Projectable:

If P: Projectable<F, W>, then the following must hold:

• Given p: *const P or p: *mut P, it is valid to perform let i = p as *const P::Inner or let i = p as *mut P::Inner. The size of the referents of p and i must be identical (e.g. as reported by size_of_val_raw).

• If the following hold:

  • p: &P or p: &mut P.
  • Given an i: P::Inner of size size_of_val(p), there exists an F at byte range f within i.

  ...then it is sound to materialize a &W or &mut W which points to range f within p.

Note that this definition holds regardless of whether P, P::Inner, or F are sized or unsized.