Deref documentation: Remove references to "smart pointers"

[e2-71828]

The documentation for Deref reads, in part:

Implementing Deref for smart pointers makes accessing the data behind them convenient, which is why they implement Deref. On the other hand, the rules regarding Deref and DerefMut were designed specifically to accommodate smart pointers. Because of this, Deref should only be implemented for smart pointers to avoid confusion.

This warning has sparked significant confusion, which seems to stem from there being no consensus of what actually is a "smart pointer". The warning should be either removed or rewritten in terms of well-understood properties.

Relevant URLO discussions:

• What are smart pointers?
• Understanding the perils of Deref

[JanBeh]

An alternative could be to actually define the term "smart pointer" somewhere. The reference contains a paragraph on smart pointers in section 4.1.13 on "pointer types", but lacks a definition.

However, it is at least questionable whether such a definition could

• reflect whether Deref should or shouldn't be implemented

and at the same time

• match the common understanding of the term (assuming there is one at all).

[ChrisDenton]

Hm from a teaching standpoint, I'd definitely agree the docs shouldn't be introducing jargon without first defining it. And in this case I'm uncertain of the benefit of introducing it at all. In Rust, is the term actually used outside of Deref?

It may be better to directly explain what kinds of types Deref should be implemented for and what would help "avoid confusion". That said, the fuzziness around the term "smart pointer" does allow leeway in interpretation, whereas a too strict definition may not properly reflect how Deref is (or should be) used in practice.

[ChrisDenton]

Follow up: The Rust book does actually have a chapter on smart pointers. And then the term is used in the next chapter on Fearless Concurrency. Some quotes from the former:

In Rust, the different smart pointers defined in the standard library provide functionality beyond that provided by references.

In Rust, which uses the concept of ownership and borrowing, an additional difference between references and smart pointers is that references are pointers that only borrow data; in contrast, in many cases, smart pointers own the data they point to.

[String and Vec<T>] count as smart pointers because they own some memory and allow you to manipulate it. They also have metadata (such as their capacity) and extra capabilities or guarantees

The characteristic that distinguishes a smart pointer from an ordinary struct is that smart pointers implement the Deref and Drop traits.

[PatchMixolydic]

The characteristic that distinguishes a smart pointer from an ordinary struct is that smart pointers implement the Deref and Drop traits.

Incidentally, String doesn't implement Drop since the inner Vec<u8>'s Drop implementation is sufficient.

[JanBeh]

An alternative could be to actually define the term "smart pointer" somewhere.

Perhaps a better way is to describe when Deref should or shouldn't be implemented. Then the term "smart pointer" or "smart reference" could (in the context of Rust) be defined as "anything that implements Deref", e.g. in the reference.

Currently some people say anything that implements Deref could be seen as smart pointer (depending on how you define it). Given the warning in Deref, this would currently be a circular definition: A smart pointer is something that implements Deref, but Deref should only be implemented for smart pointers.

If the warning was removed, the Rust community and the Rust reference could provide a clear definition what a smart pointer (or smart reference, see link above) actually is: Something that implements Deref.

[JanBeh]

Following my previous comment, the warning could be changed to something like:

"Warning: Implementing Deref on a data type will make this data type act like a smart pointer." Or: "turn it into a smart pointer." "This isn't always intended, e.g. when wrapping another type for other reasons."

[inquisitivecrystal]

I feel like actually writing out a definition of smart pointer could be helpful here. Something like "Implementing Deref will often cause instances of the implementing type to be silently converted to the target type. Please consider whether this is desirable before implementing Deref, and avoid implementing it if the distinction between the two types is important." This phrasing is awful, but maybe it captures some of what's intended?

[Ixrec]

While "smart pointer" was never a precise technical term in any language community, I think what the Deref docs are trying to get at by using the phrase is: A user/library-defined type which is generic over a type T, and which conceptually wraps a single T with indirection.

So my strawman rewrite of this paragraph:

Implementing Deref for smart pointers makes accessing the data behind them convenient, which is why they implement Deref. On the other hand, the rules regarding Deref and DerefMut were designed specifically to accommodate smart pointers. Because of this, Deref should only be implemented for smart pointers to avoid confusion.

would be something like this:

Generic wrapper types with runtime indirection such as Box<T>, Rc<T>, and Arc<T> implement Deref in order to make accessing the data behind them convenient. On the other hand, the rules regarding Deref and DerefMut were designed specifically to accommodate these pointer-like generic wrapper structs, sometimes known as "smart pointers". Because of this, Deref should only be implemented for pointer-like types to avoid confusion.

The TRPL chapter on Smart Pointers I think is acceptable, since that intro-level writing contains a lot of similar simplifications and vague concepts that an expert user would eventually discard, but we can definitely tweak the introduction to make it clearer that "smart pointer" is not a precise term. For example, changing "We’ve already encountered a few smart pointers" to "We've already encountered a few types that could be considered smart pointers".

[e2-71828]

Generic wrapper types with runtime indirection such as Box, Rc, and Arc implement Deref in order to make accessing the data behind them convenient. On the other hand, the rules regarding Deref and DerefMut were designed specifically to accommodate these pointer-like generic wrapper structs, sometimes known as "smart pointers". Because of this, Deref should only be implemented for pointer-like types to avoid confusion.

This is definitely an improvement, but it would be nice to elaborate some on "to avoid confusion:" How might this confusion manifest itself if Deref is implemented improperly?

There's also the question of whether any non-"pointer-like" types can reasonably implement Deref. Is it reasonable for a newtype that enforces some property but stores its target value inline, for example:

#[derive(Copy,Clone,Eq,PartialEq,Ord,PartialOrd,Hash,Debug)]
pub struct PrimeUsize(usize);

impl PrimeUsize {
    fn new(usize)->Self { /* Primality check here */ }
}

impl Deref for PrimeUsize {
    type Target = usize;
    fn deref(&self)->&usize { &self.0 }
}

In many ways, this behaves like String: It's a container for an owned value with some additional properties enforced by construction. Is this a reasonable use of Deref or not, and how can that be made clear in the docs?

[JanBeh]

I think what the Deref docs are trying to get at by using the phrase is: A user/library-defined type which is generic over a type T, and which conceptually wraps a single T with indirection.

I think that's an assumption that isn't necessarily true. There have been different opinions whether a "smart pointer" needs to be generic (see also the aboved linked discussion thread "What are smart pointers?"). Most prominent example is String which, according to the Rust book, is a smart pointer to str. If we would consider such non-generic types to not be smart pointers, it still makes sense to implement Deref for some of them. There are several other examples such as PathBuf (pointing/deref-ing to Path), for example. This isn't just limited to types from the standard library but might also be applicable to other types.

Introduction in Chapter 15 ("Smart Pointers"):

We’ve already encountered a few smart pointers in this book, such as String and Vec in Chapter 8, although we didn’t call them smart pointers at the time. Both these types count as smart pointers because they own some memory and allow you to manipulate it. They also have metadata (such as their capacity) and extra capabilities or guarantees (such as with String ensuring its data will always be valid UTF-8).

Also note that a String is (much) more than a wrapper to str.

[scottmcm]

Perhaps it could help to elaborate on the "confusion", rather than reference smart pointers.

For example, it could discuss how any method you add to something which implements Deref is a potential shadowing risk for something on the target. (And thus why MaybeUninit -- which doesn't Deref -- has a method assume_init_drop, but ManuallyDrop -- which does Deref -- has an associated function drop that's not a method.)