diff --git a/src/doc/rustc-dev-guide/src/ambig-unambig-ty-and-consts.md b/src/doc/rustc-dev-guide/src/ambig-unambig-ty-and-consts.md index 33b685861df37..43daf6e15ef8d 100644 --- a/src/doc/rustc-dev-guide/src/ambig-unambig-ty-and-consts.md +++ b/src/doc/rustc-dev-guide/src/ambig-unambig-ty-and-consts.md @@ -1,18 +1,18 @@ # Ambig/Unambig Types and Consts -Types and Consts args in the AST/HIR can be in two kinds of positions ambiguous (ambig) or unambiguous (unambig). Ambig positions are where -it would be valid to parse either a type or a const, unambig positions are where only one kind would be valid to -parse. +Types and Consts args in the AST/HIR can be in two kinds of positions ambiguous (ambig) or unambiguous (unambig). +Ambig positions are where it would be valid to parse either a type or a const. +Unambig positions are where only one kind would be valid to parse. ```rust fn func(arg: T) { // ^ Unambig type position - let a: _ = arg; + let a: _ = arg; // ^ Unambig type position func::(arg); // ^ ^ - // ^^^^ Ambig position + // ^^^^ Ambig position let _: [u8; 10]; // ^^ ^^ Unambig const position @@ -21,7 +21,8 @@ fn func(arg: T) { ``` -Most types/consts in ambig positions are able to be disambiguated as either a type or const during parsing. The only exceptions to this are paths and inferred generic arguments. +Most types/consts in ambig positions are able to be disambiguated as either a type or const during parsing. +The only exceptions to this are paths and inferred generic arguments. ## Paths @@ -31,7 +32,8 @@ struct Foo; fn foo(_: Foo) {} ``` -At parse time we parse all unbraced generic arguments as *types* (ie they wind up as [`ast::GenericArg::Ty`]). In the above example this means we would parse the generic argument to `Foo` as an `ast::GenericArg::Ty` wrapping a [`ast::Ty::Path(N)`]. +At parse time we parse all unbraced generic arguments as *types* (ie they wind up as [`ast::GenericArg::Ty`]). +In the above example this means we would parse the generic argument to `Foo` as an `ast::GenericArg::Ty` wrapping a [`ast::Ty::Path(N)`]. Then during name resolution: - When encountering a single segment path with no generic arguments in generic argument position, we will first try to resolve it in the type namespace and if that fails we then attempt to resolve in the value namespace. @@ -39,7 +41,8 @@ Then during name resolution: See [`LateResolutionVisitor::visit_generic_arg`] for where this is implemented. -Finally during AST lowering when we attempt to lower a type argument, we first check if it is a `Ty::Path` and if it resolved to something in the value namespace. If it did then we create an *anon const* and lower to a const argument instead of a type argument. +Finally during AST lowering when we attempt to lower a type argument, we first check if it is a `Ty::Path` and if it resolved to something in the value namespace. +If it did, then we create an *anon const* and lower to a const argument instead of a type argument. See [`LoweringContext::lower_generic_arg`] for where this is implemented. @@ -55,16 +58,26 @@ fn foo() { } ``` -The only generic arguments which remain ambiguous after lowering are inferred generic arguments (`_`) in path segments. In the above example it is not clear at parse time whether the `_` argument to `Foo` is an inferred type argument, or an inferred const argument. +The only generic arguments which remain ambiguous after lowering are inferred generic arguments (`_`) in path segments. +In the above example, +it is not clear at parse time whether the `_` argument to `Foo` is an inferred type argument, +or an inferred const argument. -In ambig AST positions, inferred argumentsd are parsed as an [`ast::GenericArg::Ty`] wrapping a [`ast::Ty::Infer`]. Then during AST lowering when lowering an `ast::GenericArg::Ty` we check if it is an inferred type and if so lower to a [`hir::GenericArg::Infer`]. +In ambig AST positions, inferred arguments are parsed as an [`ast::GenericArg::Ty`] wrapping a [`ast::Ty::Infer`]. +Then, during AST lowering, when lowering an `ast::GenericArg::Ty`, +we check if it is an inferred type, and if so, lower to a [`hir::GenericArg::Infer`]. -In unambig AST positions, inferred arguments are parsed as either `ast::Ty::Infer` or [`ast::AnonConst`]. The `AnonConst` case is quite strange, we use [`ast::ExprKind::Underscore`] to represent the "body" of the "anon const" although in reality we do not actually lower this to an anon const in the HIR. +In unambig AST positions, inferred arguments are parsed as either `ast::Ty::Infer` or [`ast::AnonConst`]. +The `AnonConst` case is quite strange; +we use [`ast::ExprKind::Underscore`] to represent the "body" of the "anon const", +although in reality we do not actually lower this to an anon const in the HIR. -It may be worth seeing if we can refactor the AST to have `ast::GenericArg::Infer` and then get rid of this overloaded meaning of `AnonConst`, as well as the reuse of `ast::Ty::Infer` in ambig positions. +It may be worth seeing if we can refactor the AST to have `ast::GenericArg::Infer` and then get rid of this overloaded meaning of `AnonConst`, +as well as the reuse of `ast::Ty::Infer` in ambig positions. In unambig AST positions, during AST lowering we lower inferred arguments to [`hir::TyKind::Infer`][ty_infer] or [`hir::ConstArgKind::Infer`][const_infer] depending on whether it is a type or const position respectively. -In ambig AST positions, during AST lowering we lower inferred arguments to [`hir::GenericArg::Infer`][generic_arg_infer]. See [`LoweringContext::lower_generic_arg`] for where this is implemented. +In ambig AST positions, during AST lowering we lower inferred arguments to [`hir::GenericArg::Infer`][generic_arg_infer]. +See [`LoweringContext::lower_generic_arg`] for where this is implemented. A naive implementation of this would result in there being potentially 5 places where you might think an inferred type/const could be found in the HIR from looking at the structure of the HIR: 1. In unambig type position as a `TyKind::Infer` @@ -73,7 +86,7 @@ A naive implementation of this would result in there being potentially 5 places 4. In an ambig position as a [`GenericArg::Const(ConstArgKind::Infer)`][generic_arg_const] 5. In an ambig position as a `GenericArg::Infer` -Note that places 3 and 4 would never actually be possible to encounter as we always lower to `GenericArg::Infer` in generic arg position. +Note that places 3 and 4 would never actually be possible to encounter as we always lower to `GenericArg::Infer` in generic arg position. This has a few failure modes: - People may write visitors which check for `GenericArg::Infer` but forget to check for `hir::TyKind/ConstArgKind::Infer`, only handling infers in ambig positions by accident. @@ -83,13 +96,15 @@ This has a few failure modes: To make writing HIR visitors less error prone when caring about inferred types/consts we have a relatively complex system: -1. We have different types in the compiler for when a type or const is in an unambig or ambig position, `Ty` and `Ty<()>`. [`AmbigArg`][ambig_arg] is an uninhabited type which we use in the `Infer` variant of `TyKind` and `ConstArgKind` to selectively "disable" it if we are in an ambig position. +1. We have different types in the compiler for when a type or const is in an unambig or ambig position, `Ty` and `Ty<()>`. + [`AmbigArg`][ambig_arg] is an uninhabited type which we use in the `Infer` variant of `TyKind` and `ConstArgKind` to selectively "disable" it if we are in an ambig position. -2. The [`visit_ty`][visit_ty] and [`visit_const_arg`][visit_const_arg] methods on HIR visitors only accept the ambig position versions of types/consts. Unambig types/consts are implicitly converted to ambig types/consts during the visiting process, with the `Infer` variant handled by a dedicated [`visit_infer`][visit_infer] method. +2. The [`visit_ty`][visit_ty] and [`visit_const_arg`][visit_const_arg] methods on HIR visitors only accept the ambig position versions of types/consts. + Unambig types/consts are implicitly converted to ambig types/consts during the visiting process, with the `Infer` variant handled by a dedicated [`visit_infer`][visit_infer] method. This has a number of benefits: - It's clear that `GenericArg::Type/Const` cannot represent inferred type/const arguments -- Implementors of `visit_ty` and `visit_const_arg` will never encounter inferred types/consts making it impossible to write a visitor that seems to work right but handles edge cases wrong +- Implementors of `visit_ty` and `visit_const_arg` will never encounter inferred types/consts making it impossible to write a visitor that seems to work right but handles edge cases wrong - The `visit_infer` method handles *all* cases of inferred type/consts in the HIR making it easy for visitors to handle inferred type/consts in one dedicated place and not forget cases [ty_infer]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/enum.TyKind.html#variant.Infer diff --git a/src/doc/rustc-dev-guide/src/autodiff/installation.md b/src/doc/rustc-dev-guide/src/autodiff/installation.md index 4176763a7bd62..c0da9621f9777 100644 --- a/src/doc/rustc-dev-guide/src/autodiff/installation.md +++ b/src/doc/rustc-dev-guide/src/autodiff/installation.md @@ -3,7 +3,7 @@ Most users can enable `std::autodiff` on their latest nightly toolchain by installing the `enzyme` component with rustup, if they are using one of these platforms: - **Linux**: with `x86_64-unknown-linux-gnu` or `aarch64-unknown-linux-gnu` -- **macOS**: with `aarch64-apple-darwin` +- **macOS**: with `aarch64-apple-darwin` or `x86_64-apple-darwin` - **Windows**: with `x86_64-llvm-mingw` or `aarch64-llvm-mingw` As a rustc/enzyme/autodiff contributor, or if you need any other platform, you can build rustc including autodiff from source. @@ -18,6 +18,12 @@ Please run: rustup +nightly component add enzyme ``` +Older rustup versions are not aware of this component, so if you run into issues try updating rustup itself: +```console +rustup update +rustup +nightly component add enzyme +``` + ## Installation guide for Nix On [Nix], you can declare a nightly Rust toolchain with the Enzyme component using the [oxalica rust-overlay]. diff --git a/src/doc/rustc-dev-guide/src/coroutine-closures.md b/src/doc/rustc-dev-guide/src/coroutine-closures.md index 45d6a24a65998..0e917c4300bab 100644 --- a/src/doc/rustc-dev-guide/src/coroutine-closures.md +++ b/src/doc/rustc-dev-guide/src/coroutine-closures.md @@ -1,6 +1,10 @@ # Async closures/"coroutine-closures" -Please read [RFC 3668](https://rust-lang.github.io/rfcs/3668-async-closures.html) to understand the general motivation of the feature. This is a very technical and somewhat "vertical" chapter; ideally we'd split this and sprinkle it across all the relevant chapters, but for the purposes of understanding async closures *holistically*, I've put this together all here in one chapter. +Please read [RFC 3668] to understand the general motivation of the feature. +This is a very technical and somewhat "vertical" chapter; +ideally, we'd split this and sprinkle it across all the relevant chapters, +but for the purposes of understanding async closures *holistically*, +I've put this together all here in one chapter. ## Coroutine-closures -- a technical deep dive @@ -8,7 +12,8 @@ Coroutine-closures are a generalization of async closures, being special syntax For now, the only usable kind of coroutine-closure is the async closure, and supporting async closures is the extent of this PR. We may eventually support `gen || {}`, etc., and most of the problems and curiosities described in this document apply to all coroutine-closures in general. -As a consequence of the code being somewhat general, this document may flip between calling them "async closures" and "coroutine-closures". The future that is returned by the async closure will generally be called the "coroutine" or the "child coroutine". +As a consequence of the code being somewhat general, this document may flip between calling them "async closures" and "coroutine-closures". +The future that is returned by the async closure will generally be called the "coroutine" or the "child coroutine". ### HIR @@ -20,7 +25,10 @@ The closure-kind of the async block is `ClosureKind::Closure(CoroutineKind::Desu [^k2]: -Like `async fn`, when lowering an async closure's body, we need to unconditionally move all of the closures arguments into the body so they are captured. This is handled by `lower_coroutine_body_with_moved_arguments`[^l1]. The only notable quirk with this function is that the async block we end up generating as a capture kind of `CaptureBy::ByRef`[^l2]. We later force all of the *closure args* to be captured by-value[^l3], but we don't want the *whole* async block to act as if it were an `async move`, since that would defeat the purpose of the self-borrowing of an async closure. +Like `async fn`, when lowering an async closure's body, we need to unconditionally move all of the closures arguments into the body so they are captured. +This is handled by `lower_coroutine_body_with_moved_arguments`[^l1]. +The only notable quirk with this function is that the async block we end up generating as a capture kind of `CaptureBy::ByRef`[^l2]. +We later force all of the *closure args* to be captured by-value[^l3], but we don't want the *whole* async block to act as if it were an `async move`, since that would defeat the purpose of the self-borrowing of an async closure. [^l1]: @@ -28,7 +36,7 @@ Like `async fn`, when lowering an async closure's body, we need to unconditional [^l3]: -### `rustc_middle::ty` Representation +### `rustc_middle::ty` representation For the purposes of keeping the implementation mostly future-compatible (i.e. with gen `|| {}` and `async gen || {}`), most of this section calls async closures "coroutine-closures". @@ -36,7 +44,8 @@ The main thing that this PR introduces is a new `TyKind` called `CoroutineClosur [^t1]: -We introduce a new `TyKind` instead of generalizing the existing `TyKind::Closure` due to major representational differences in the type. The major differences between `CoroutineClosure`s can be explored by first inspecting the `CoroutineClosureArgsParts`, which is the "unpacked" representation of the coroutine-closure's generics. +We introduce a new `TyKind` instead of generalizing the existing `TyKind::Closure` due to major representational differences in the type. +The major differences between `CoroutineClosure`s can be explored by first inspecting the `CoroutineClosureArgsParts`, which is the "unpacked" representation of the coroutine-closure's generics. #### Similarities to closures @@ -44,11 +53,13 @@ Like a closure, we have `parent_args`, a `closure_kind_ty`, and a `tupled_upvars #### The signature -A traditional closure has a `fn_sig_as_fn_ptr_ty` which it uses to represent the signature of the closure. In contrast, we store the signature of a coroutine closure in a somewhat "exploded" way, since coroutine-closures have *two* signatures depending on what `AsyncFn*` trait you call it with (see below sections). +A traditional closure has a `fn_sig_as_fn_ptr_ty` which it uses to represent the signature of the closure. +In contrast, we store the signature of a coroutine closure in a somewhat "exploded" way, since coroutine-closures have *two* signatures depending on what `AsyncFn*` trait you call it with (see below sections). Conceptually, the coroutine-closure may be thought as containing several different signature types depending on whether it is being called by-ref or by-move. -To conveniently recreate both of these signatures, the `signature_parts_ty` stores all of the relevant parts of the coroutine returned by this coroutine-closure. This signature parts type will have the general shape of `fn(tupled_inputs, resume_ty) -> (return_ty, yield_ty)`, where `resume_ty`, `return_ty`, and `yield_ty` are the respective types for the *coroutine* returned by the coroutine-closure[^c1]. +To conveniently recreate both of these signatures, the `signature_parts_ty` stores all of the relevant parts of the coroutine returned by this coroutine-closure. +This signature parts type will have the general shape of `fn(tupled_inputs, resume_ty) -> (return_ty, yield_ty)`, where `resume_ty`, `return_ty`, and `yield_ty` are the respective types for the *coroutine* returned by the coroutine-closure[^c1]. [^c1]: @@ -76,9 +87,10 @@ To most easily construct the `Coroutine` that a coroutine-closure returns, you c Most of the args to that function will be components that you can get out of the `CoroutineArgs`, except for the `goal_kind: ClosureKind` which controls which flavor of coroutine to return based off of the `ClosureKind` passed in -- i.e. it will prepare the by-ref coroutine if `ClosureKind::Fn | ClosureKind::FnMut`, and the by-move coroutine if `ClosureKind::FnOnce`. -### Trait Hierarchy +### Trait hierarchy -We introduce a parallel hierarchy of `Fn*` traits that are implemented for . The motivation for the introduction was covered in a blog post: [Async Closures](https://hackmd.io/@compiler-errors/async-closures). +We introduce a parallel hierarchy of `Fn*` traits that are implemented for . +The motivation for the introduction was covered in a blog post: [Async Closures](https://hackmd.io/@compiler-errors/async-closures). All currently-stable callable types (i.e., closures, function items, function pointers, and `dyn Fn*` trait objects) automatically implement `AsyncFn*() -> T` if they implement `Fn*() -> Fut` for some output type `Fut`, and `Fut` implements `Future`[^tr1]. @@ -88,23 +100,27 @@ Async closures implement `AsyncFn*` as their bodies permit; i.e. if they end up #### Lending -We may in the future move `AsyncFn*` onto a more general set of `LendingFn*` traits; however, there are some concrete technical implementation details that limit our ability to use `LendingFn` ergonomically in the compiler today. These have to do with: +We may in the future move `AsyncFn*` onto a more general set of `LendingFn*` traits; however, there are some concrete technical implementation details that limit our ability to use `LendingFn` ergonomically in the compiler today. +These have to do with: - Closure signature inference. - Limitations around higher-ranked trait bounds. - Shortcomings with error messages. -These limitations, plus the fact that the underlying trait should have no effect on the user experience of async closures and async `Fn` trait bounds, leads us to `AsyncFn*` for now. To ensure we can eventually move to these more general traits, the precise `AsyncFn*` trait definitions (including the associated types) are left as an implementation detail. +These limitations, plus the fact that the underlying trait should have no effect on the user experience of async closures and async `Fn` trait bounds, leads us to `AsyncFn*` for now. +To ensure we can eventually move to these more general traits, the precise `AsyncFn*` trait definitions (including the associated types) are left as an implementation detail. #### When do async closures implement the regular `Fn*` traits? We mention above that "regular" callable types can implement `AsyncFn*`, but the reverse question exists of "can async closures implement `Fn*` too"? The short answer is "when it's valid", i.e. when the coroutine that would have been returned from `AsyncFn`/`AsyncFnMut` does not actually have any upvars that are "lent" from the parent coroutine-closure. -See the "follow-up: when do..." section below for an elaborated answer. The full answer describes a pretty interesting and hopefully thorough heuristic that is used to ensure that most async closures "just work". +See the "follow-up: when do..." section below for an elaborated answer. +The full answer describes a pretty interesting and hopefully thorough heuristic that is used to ensure that most async closures "just work". ### Tale of two bodies... -When async closures are called with `AsyncFn`/`AsyncFnMut`, they return a coroutine that borrows from the closure. However, when they are called via `AsyncFnOnce`, we consume that closure, and cannot return a coroutine that borrows from data that is now dropped. +When async closures are called with `AsyncFn`/`AsyncFnMut`, they return a coroutine that borrows from the closure. +However, when they are called via `AsyncFnOnce`, we consume that closure, and cannot return a coroutine that borrows from data that is now dropped. To work around this limitation, we synthesize a separate by-move MIR body for calling `AsyncFnOnce::call_once` on a coroutine-closure that can be called by-ref. @@ -120,18 +136,24 @@ This query synthesizes a new MIR body by copying the MIR body of the coroutine a [^b2]: -Since we've synthesized a new def id, this query is also responsible for feeding a ton of other relevant queries for the MIR body. This query is `ensure()`d[^b3] during the `mir_promoted` query, since it operates on the *built* mir of the coroutine. +Since we've synthesized a new def id, this query is also responsible for feeding a ton of other relevant queries for the MIR body. +This query is `ensure()`d[^b3] during the `mir_promoted` query, since it operates on the *built* mir of the coroutine. [^b3]: ### Closure signature inference -The closure signature inference algorithm for async closures is a bit more complicated than the inference algorithm for "traditional" closures. Like closures, we iterate through all of the clauses that may be relevant (for the expectation type passed in)[^deduce1]. +The closure signature inference algorithm for async closures is a bit more complicated than the inference algorithm for "traditional" closures. +Like closures, we iterate through all of the clauses that may be relevant (for the expectation type passed in)[^deduce1]. To extract a signature, we consider two situations: -* Projection predicates with `AsyncFnOnce::Output`, which we will use to extract the inputs and output type for the closure. This corresponds to the situation that there was a `F: AsyncFn*() -> T` bound[^deduce2]. -* Projection predicates with `FnOnce::Output`, which we will use to extract the inputs. For the output, we also try to deduce an output by looking for relevant `Future::Output` projection predicates. This corresponds to the situation that there was an `F: Fn*() -> T, T: Future` bound.[^deduce3] - * If there is no `Future` bound, we simply use a fresh infer var for the output. This corresponds to the case where one can pass an async closure to a combinator function like `Option::map`.[^deduce4] +* Projection predicates with `AsyncFnOnce::Output`, which we will use to extract the inputs and output type for the closure. + This corresponds to the situation that there was a `F: AsyncFn*() -> T` bound[^deduce2]. +* Projection predicates with `FnOnce::Output`, which we will use to extract the inputs. + For the output, we also try to deduce an output by looking for relevant `Future::Output` projection predicates. + This corresponds to the situation that there was an `F: Fn*() -> T, T: Future` bound.[^deduce3] + * If there is no `Future` bound, we simply use a fresh infer var for the output. + This corresponds to the case where one can pass an async closure to a combinator function like `Option::map`.[^deduce4] [^deduce1]: @@ -149,7 +171,7 @@ We defer[^call1] the computation of a coroutine-closure's "kind" (i.e. its maxim [^call1]: -Unlike regular closures, whose return type does not change depending on what `Fn*` trait we call it with, coroutine-closures *do* end up returning different coroutine types depending on the flavor of `AsyncFn*` trait used to call it. +Unlike regular closures, whose return type does not change depending on what `Fn*` trait we call it with, coroutine-closures *do* end up returning different coroutine types depending on the flavor of `AsyncFn*` trait used to call it. Specifically, while the def-id of the returned coroutine does not change, the upvars[^call2] (which are either borrowed or moved from the parent coroutine-closure) and the coroutine-kind[^call3] are dependent on the calling mode. @@ -165,7 +187,9 @@ We introduce a `AsyncFnKindHelper` trait which allows us to defer the question o #### Ok, so why? -This seems a bit roundabout and complex, and I admit that it is. But let's think of the "do nothing" alternative -- we could instead mark all `AsyncFn*` goals as ambiguous until upvar analysis, at which point we would know exactly what to put into the upvars of the coroutine we return. However, this is actually *very* detrimental to inference in the program, since it means that programs like this would not be valid: +This seems a bit roundabout and complex, and I admit that it is. +But let's think of the "do nothing" alternative -- we could instead mark all `AsyncFn*` goals as ambiguous until upvar analysis, at which point we would know exactly what to put into the upvars of the coroutine we return. +However, this is actually *very* detrimental to inference in the program, since it means that programs like this would not be valid: ```rust,ignore let c = async || -> String { .. }; @@ -179,27 +203,35 @@ So *instead*, we use this alias (in this case, a projection: `AsyncFnKindHelper: ### Upvar analysis -By and large, the upvar analysis for coroutine-closures and their child coroutines proceeds like normal upvar analysis. However, there are several interesting bits that happen to account for async closures' special natures: +By and large, the upvar analysis for coroutine-closures and their child coroutines proceeds like normal upvar analysis. +However, there are several interesting bits that happen to account for async closures' special natures: #### Forcing all inputs to be captured -Like async fn, all input arguments are captured. We explicitly force[^f1] all of these inputs to be captured by move so that the future coroutine returned by async closures does not depend on whether the input is *used* by the body or not, which would impart an interesting semver hazard. +Like async fn, all input arguments are captured. +We explicitly force[^f1] all of these inputs to be captured by move so that the future coroutine returned by async closures does not depend on whether the input is *used* by the body or not, which would impart an interesting semver hazard. [^f1]: #### Computing the by-ref captures -For a coroutine-closure that supports `AsyncFn`/`AsyncFnMut`, we must also compute the relationship between the captures of the coroutine-closure and its child coroutine. Specifically, the coroutine-closure may `move` a upvar into its captures, but the coroutine may only borrow that upvar. +For a coroutine-closure that supports `AsyncFn`/`AsyncFnMut`, we must also compute the relationship between the captures of the coroutine-closure and its child coroutine. +Specifically, the coroutine-closure may `move` a upvar into its captures, but the coroutine may only borrow that upvar. -We compute the "`coroutine_captures_by_ref_ty`" by looking at all of the child coroutine's captures and comparing them to the corresponding capture of the parent coroutine-closure[^br1]. This `coroutine_captures_by_ref_ty` ends up being represented as a `for<'env> fn() -> captures...` type, with the additional binder lifetime representing the "`&self`" lifetime of calling `AsyncFn::async_call` or `AsyncFnMut::async_call_mut`. We instantiate that binder later when actually calling the methods. +We compute the "`coroutine_captures_by_ref_ty`" by looking at all of the child coroutine's captures and comparing them to the corresponding capture of the parent coroutine-closure[^br1]. +This `coroutine_captures_by_ref_ty` ends up being represented as a `for<'env> fn() -> captures...` type, with the additional binder lifetime representing the "`&self`" lifetime of calling `AsyncFn::async_call` or `AsyncFnMut::async_call_mut`. +We instantiate that binder later when actually calling the methods. [^br1]: -Note that not every by-ref capture from the parent coroutine-closure results in a "lending" borrow. See the **Follow-up: When do async closures implement the regular `Fn*` traits?** section below for more details, since this intimately influences whether or not the coroutine-closure is allowed to implement the `Fn*` family of traits. +Note that not every by-ref capture from the parent coroutine-closure results in a "lending" borrow. +See the **Follow-up: When do async closures implement the regular `Fn*` traits?** section below for more details, since this intimately influences whether or not the coroutine-closure is allowed to implement the `Fn*` family of traits. #### By-move body + `FnOnce` quirk -There are several situations where the closure upvar analysis ends up inferring upvars for the coroutine-closure's child coroutine that are too relaxed, and end up resulting in borrow-checker errors. This is best illustrated via examples. For example, given: +There are several situations where the closure upvar analysis ends up inferring upvars for the coroutine-closure's child coroutine that are too relaxed, and end up resulting in borrow-checker errors. +This is best illustrated via examples. +For example, given: ```rust fn force_fnonce(t: T) -> T { t } @@ -210,7 +242,8 @@ let c = force_fnonce(async move || { }); ``` -`x` will be moved into the coroutine-closure, but the coroutine that is returned would only borrow `&x`. However, since `force_fnonce` forces the coroutine-closure to `AsyncFnOnce`, which is not *lending*, we must force the capture to happen by-move[^bm1]. +`x` will be moved into the coroutine-closure, but the coroutine that is returned would only borrow `&x`. +However, since `force_fnonce` forces the coroutine-closure to `AsyncFnOnce`, which is not *lending*, we must force the capture to happen by-move[^bm1]. Similarly: @@ -233,13 +266,16 @@ let c = async move || { Well, first of all, all async closures implement `FnOnce` since they can always be called *at least once*. -For `Fn`/`FnMut`, the detailed answer involves answering a related question: is the coroutine-closure lending? Because if it is, then it cannot implement the non-lending `Fn`/`FnMut` traits. +For `Fn`/`FnMut`, the detailed answer involves answering a related question: is the coroutine-closure lending? +Because if it is, then it cannot implement the non-lending `Fn`/`FnMut` traits. -Determining when the coroutine-closure must *lend* its upvars is implemented in the `should_reborrow_from_env_of_parent_coroutine_closure` helper function[^u1]. Specifically, this needs to happen in two places: +Determining when the coroutine-closure must *lend* its upvars is implemented in the `should_reborrow_from_env_of_parent_coroutine_closure` helper function[^u1]. +Specifically, this needs to happen in two places: [^u1]: -1. Are we borrowing data owned by the parent closure? We can determine if that is the case by checking if the parent capture is by-move, EXCEPT if we apply a deref projection, which means we're reborrowing a reference that we captured by-move. +1. Are we borrowing data owned by the parent closure? + We can determine if that is the case by checking if the parent capture is by-move, EXCEPT if we apply a deref projection, which means we're reborrowing a reference that we captured by-move. ```rust let x = &1i32; // Let's call this lifetime `'1`. @@ -250,7 +286,8 @@ let c = async move || { }; ``` -2. If a coroutine is mutably borrowing from a parent capture, then that mutable borrow cannot live for longer than either the parent *or* the borrow that we have on the original upvar. Therefore we always need to borrow the child capture with the lifetime of the parent coroutine-closure's env. +2. If a coroutine is mutably borrowing from a parent capture, then that mutable borrow cannot live for longer than either the parent *or* the borrow that we have on the original upvar. + Therefore we always need to borrow the child capture with the lifetime of the parent coroutine-closure's env. ```rust let mut x = 1i32; @@ -264,7 +301,8 @@ let c = async || { }; ``` -If either of these cases apply, then we should capture the borrow with the lifetime of the parent coroutine-closure's env. Luckily, if this function is not correct, then the program is not unsound, since we still borrowck and validate the choices made from this function -- the only side-effect is that the user may receive unnecessary borrowck errors. +If either of these cases apply, then we should capture the borrow with the lifetime of the parent coroutine-closure's env. +Luckily, if this function is not correct, then the program is not unsound, since we still borrowck and validate the choices made from this function -- the only side-effect is that the user may receive unnecessary borrowck errors. ### Instance resolution @@ -278,7 +316,8 @@ If a coroutine-closure has a closure-kind of `FnMut`/`Fn`, then the same applies [^res3]: -This is represented by the `ConstructCoroutineInClosureShim`[^i1]. The `receiver_by_ref` bool will be true if this is the instance of `Fn::call`/`FnMut::call_mut`.[^i2] The coroutine that all of these instances returns corresponds to the by-move body we will have synthesized by this point.[^i3] +This is represented by the `ConstructCoroutineInClosureShim`[^i1]. +The `receiver_by_ref` bool will be true if this is the instance of `Fn::call`/`FnMut::call_mut`.[^i2] The coroutine that all of these instances returns corresponds to the by-move body we will have synthesized by this point.[^i3] [^i1]: @@ -288,10 +327,13 @@ This is represented by the `ConstructCoroutineInClosureShim`[^i1]. The `receiver ### Borrow-checking -It turns out that borrow-checking async closures is pretty straightforward. After adding a new `DefiningTy::CoroutineClosure`[^bck1] variant, and teaching borrowck how to generate the signature of the coroutine-closure[^bck2], borrowck proceeds totally fine. +It turns out that borrow-checking async closures is pretty straightforward. +After adding a new `DefiningTy::CoroutineClosure`[^bck1] variant, and teaching borrowck how to generate the signature of the coroutine-closure[^bck2], borrowck proceeds totally fine. One thing to note is that we don't borrow-check the synthetic body we make for by-move coroutines, since by construction (and the validity of the by-ref coroutine body it was derived from) it must be valid. [^bck1]: [^bck2]: + +[RFC 3668]: https://rust-lang.github.io/rfcs/3668-async-closures.html diff --git a/src/doc/rustc-dev-guide/src/diagnostics.md b/src/doc/rustc-dev-guide/src/diagnostics.md index 7e3da67d872c1..651f86329c9e3 100644 --- a/src/doc/rustc-dev-guide/src/diagnostics.md +++ b/src/doc/rustc-dev-guide/src/diagnostics.md @@ -339,19 +339,18 @@ For example, Most of these methods will accept strings, but it is recommended that typed identifiers for translatable diagnostics be used for new diagnostics (see -[Translation][translation]). +[Translation]). [translation]: ./diagnostics/translation.md `Diag` allows you to add related notes and suggestions to an error -before emitting it by calling the [`emit`][emit] method. +before emitting it by calling the [`emit`] method. (Failing to either emit or [cancel] a `Diag` will result in an ICE.) See the [docs][diag] for more info on what you can do. [spanerr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.DiagCtxt.html#method.span_err [strspanerr]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.DiagCtxt.html#method.struct_span_err -[diag]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.Diag.html -[emit]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.Diag.html#method.emit +[`emit`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.Diag.html#method.emit [cancel]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.Diag.html#method.cancel ```rust,ignore @@ -390,12 +389,11 @@ To this end, suggestions pleasingly in the terminal, or (when the `--error-format json` flag is passed) as JSON for consumption by tools like [`rustfix`][rustfix]. -[diag]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.Diag.html [rustfix]: https://github.com/rust-lang/rustfix -Not all suggestions should be applied mechanically, they have a degree of -confidence in the suggested code, from high -(`Applicability::MachineApplicable`) to low (`Applicability::MaybeIncorrect`). +Not all suggestions should be applied mechanically; +they have a degree of confidence in the suggested code, +from high (`Applicability::MachineApplicable`) to low (`Applicability::MaybeIncorrect`). Be conservative when choosing the level. Use the [`span_suggestion`][span_suggestion] method of `Diag` to make a suggestion. @@ -1049,3 +1047,5 @@ Will format the message into ```text "Self = `i8`, T = `i32`, this = `From`, trait = `From`, context = `a function`" ``` + +[diag]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_errors/struct.Diag.html diff --git a/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-items.md b/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-items.md index 9360427d660e2..61d571a5d8cd9 100644 --- a/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-items.md +++ b/src/doc/rustc-dev-guide/src/diagnostics/diagnostic-items.md @@ -1,19 +1,20 @@ # Diagnostic Items -While writing lints it's common to check for specific types, traits and -functions. This raises the question on how to check for these. Types can be -checked by their complete type path. However, this requires hard coding paths -and can lead to misclassifications in some edge cases. To counteract this, -rustc has introduced diagnostic items that are used to identify types via -[`Symbol`]s. +While writing lints, it's common to check for specific types, traits, and functions. +This raises the question on how to check for these. +Types can be checked by their complete type path. +However, this requires hard coding paths and can lead to misclassifications in some edge cases. +To counteract this, +rustc has introduced diagnostic items that are used to identify types via [`Symbol`]s. ## Finding diagnostic items Diagnostic items are added to items inside `rustc`/`std`/`core`/`alloc` with the -`rustc_diagnostic_item` attribute. The item for a specific type can be found by +`rustc_diagnostic_item` attribute. +The item for a specific type can be found by opening the source code in the documentation and looking for this attribute. -Note that it's often added with the `cfg_attr` attribute to avoid compilation -errors during tests. A definition often looks like this: +Note that it's often added with the `cfg_attr` attribute to avoid compilation errors during tests. +A definition often looks like this: ```rs // This is the diagnostic item for this type vvvvvvv @@ -32,12 +33,13 @@ please use the diagnostic item of the item and reference A new diagnostic item can be added with these two steps: -1. Find the target item inside the Rust repo. Now add the diagnostic item as a - string via the `rustc_diagnostic_item` attribute. This can sometimes cause - compilation errors while running tests. These errors can be avoided by using +1. Find the target item inside the Rust repo. + Now add the diagnostic item as a string via the `rustc_diagnostic_item` attribute. + This can sometimes cause compilation errors while running tests. + These errors can be avoided by using the `cfg_attr` attribute with the `not(test)` condition (it's fine adding - then for all `rustc_diagnostic_item` attributes as a preventive manner). At - the end, it should look like this: + then for all `rustc_diagnostic_item` attributes as a preventive manner). + At the end, it should look like this: ```rs // This will be the new diagnostic item vvv @@ -48,14 +50,13 @@ A new diagnostic item can be added with these two steps: For the naming conventions of diagnostic items, please refer to [*Naming Conventions*](#naming-conventions). -2. - Diagnostic items in code are accessed via symbols in - [`rustc_span::symbol::sym`]. +2. + Diagnostic items in code are accessed via symbols in [`rustc_span::symbol::sym`]. To add your newly-created diagnostic item, simply open the module file, - and add the name (In this case `Cat`) at the correct point in the list. + and add the name, in this case `Cat`, at the correct point in the list. -Now you can create a pull request with your changes. :tada: +Now you can create a pull request with your changes. > NOTE: > When using diagnostic items in other projects like Clippy, @@ -67,19 +68,15 @@ Diagnostic items don't have a naming convention yet. Following are some guidelines that should be used in future, but might differ from existing names: -* Types, traits, and enums are named using UpperCamelCase - (Examples: `Iterator` and `HashMap`) +* Types, traits, and enums are named using UpperCamelCase (Examples: `Iterator` and `HashMap`) * For type names that are used multiple times, like `Writer`, it's good to choose a more precise name, - maybe by adding the module to it - (Example: `IoWriter`) + maybe by adding the module to it (Example: `IoWriter`) * Associated items should not get their own diagnostic items, - but instead be accessed indirectly by the diagnostic item - of the type they're originating from. + but instead be accessed indirectly by the diagnostic item of the type they're originating from. * Freestanding functions like `std::mem::swap()` should be named using - `snake_case` with one important (export) module as a prefix - (Examples: `mem_swap` and `cmp_max`) + `snake_case` with one important (export) module as a prefix (Examples: `mem_swap` and `cmp_max`) * Modules should usually not have a diagnostic item attached to them. Diagnostic items were added to avoid the usage of paths, and using them on modules would therefore most likely be counterproductive. @@ -87,11 +84,12 @@ but might differ from existing names: ## Using diagnostic items In rustc, diagnostic items are looked up via [`Symbol`]s from inside the -[`rustc_span::symbol::sym`] module. These can then be mapped to [`DefId`]s +[`rustc_span::symbol::sym`] module. +These can then be mapped to [`DefId`]s using [`TyCtxt::get_diagnostic_item()`] or checked if they match a [`DefId`] -using [`TyCtxt::is_diagnostic_item()`]. When mapping from a diagnostic item to -a [`DefId`], the method will return a `Option`. This can be `None` if -either the symbol isn't a diagnostic item or the type is not registered, for +using [`TyCtxt::is_diagnostic_item()`]. +When mapping from a diagnostic item to a [`DefId`], the method will return a `Option`. +This can be `None` if either the symbol isn't a diagnostic item or the type is not registered, for instance when compiling with `#[no_std]`. All the following examples are based on [`DefId`]s and their usage. @@ -130,22 +128,20 @@ fn is_diag_trait_item( ### Associated Types Associated types of diagnostic items can be accessed indirectly by first -getting the [`DefId`] of the trait and then calling -[`TyCtxt::associated_items()`]. This returns an [`AssocItems`] object which can -be used for further checks. Checkout -[`clippy_utils::ty::get_iterator_item_ty()`] for an example usage of this. +getting the [`DefId`] of the trait and then calling [`TyCtxt::associated_items()`]. +This returns an [`AssocItems`] object which can be used for further checks. +Checkout [`clippy_utils::ty::get_iterator_item_ty()`] for an example usage of this. ### Usage in Clippy Clippy tries to use diagnostic items where possible and has developed some -wrapper and utility functions. Please also refer to its documentation when -using diagnostic items in Clippy. (See [*Common tools for writing -lints*][clippy-Common-tools-for-writing-lints].) +wrapper and utility functions. +Please also refer to its documentation when using diagnostic items in Clippy. +(See [*Common tools for writing lints*].) ## Related issues -These are probably only interesting to people -who really want to take a deep dive into the topic :) +These are probably only interesting to people who really want to take a deep dive into the topic :) * [rust#60966]: The Rust PR that introduced diagnostic items * [rust-clippy#5393]: Clippy's tracking issue for moving away from hard coded paths to @@ -161,6 +157,6 @@ who really want to take a deep dive into the topic :) [`TyCtxt::associated_items()`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.associated_items [`AssocItems`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/assoc/struct.AssocItems.html [`clippy_utils::ty::get_iterator_item_ty()`]: https://github.com/rust-lang/rust-clippy/blob/305177342fbc622c0b3cb148467bab4b9524c934/clippy_utils/src/ty.rs#L55-L72 -[clippy-Common-tools-for-writing-lints]: https://doc.rust-lang.org/nightly/clippy/development/common_tools_writing_lints.html +[*Common tools for writing lints*]: https://doc.rust-lang.org/nightly/clippy/development/common_tools_writing_lints.html [rust#60966]: https://github.com/rust-lang/rust/pull/60966 [rust-clippy#5393]: https://github.com/rust-lang/rust-clippy/issues/5393 diff --git a/src/doc/rustc-dev-guide/src/mir/dataflow.md b/src/doc/rustc-dev-guide/src/mir/dataflow.md index 970e61196c122..651de63e54a04 100644 --- a/src/doc/rustc-dev-guide/src/mir/dataflow.md +++ b/src/doc/rustc-dev-guide/src/mir/dataflow.md @@ -1,51 +1,52 @@ # Dataflow Analysis If you work on the MIR, you will frequently come across various flavors of -[dataflow analysis][wiki]. `rustc` uses dataflow to find uninitialized +[dataflow analysis]. +`rustc` uses dataflow to find uninitialized variables, determine what variables are live across a generator `yield` -statement, and compute which `Place`s are borrowed at a given point in the -control-flow graph. Dataflow analysis is a fundamental concept in modern -compilers, and knowledge of the subject will be helpful to prospective -contributors. +statement, and compute which `Place`s are borrowed at a given point in the control-flow graph. +Dataflow analysis is a fundamental concept in modern +compilers, and knowledge of the subject will be helpful to prospective contributors. However, this documentation is not a general introduction to dataflow analysis. -It is merely a description of the framework used to define these analyses in -`rustc`. It assumes that the reader is familiar with the core ideas as well as +It is merely a description of the framework used to define these analyses in `rustc`. +It assumes that the reader is familiar with the core ideas as well as some basic terminology, such as "transfer function", "fixpoint" and "lattice". If you're unfamiliar with these terms, or if you want a quick refresher, -[*Static Program Analysis*] by Anders Møller and Michael I. Schwartzbach is an -excellent, freely available textbook. For those who prefer audiovisual -learning, we previously recommended a series of short lectures +[*Static Program Analysis*] is an excellent and freely available textbook. +For those who prefer audiovisual learning, we previously recommended a series of short lectures by the Goethe University Frankfurt on YouTube, but it has since been deleted. See [this PR][pr-1295] for the context and [this comment][pr-1295-comment] for the alternative lectures. ## Defining a Dataflow Analysis -A dataflow analysis is defined by the [`Analysis`] trait. In addition to the -type of the dataflow state, this trait defines the initial value of that state -at entry to each block, as well as the direction of the analysis, either -forward or backward. The domain of your dataflow analysis must be a [lattice][] -(strictly speaking a join-semilattice) with a well-behaved `join` operator. See -documentation for the [`lattice`] module, as well as the [`JoinSemiLattice`] +A dataflow analysis is defined by the [`Analysis`] trait. +In addition to the type of the dataflow state, this trait defines the initial value of that state +at entry to each block, as well as the direction of the analysis, either forward or backward. +The domain of your dataflow analysis must be a [lattice] +(strictly speaking a join-semilattice) with a well-behaved `join` operator. +See documentation for the [`lattice`] module, as well as the [`JoinSemiLattice`] trait, for more information. ### Transfer Functions and Effects The dataflow framework in `rustc` allows each statement (and terminator) inside -a basic block to define its own transfer function. For brevity, these -individual transfer functions are known as "effects". Each effect is applied -successively in dataflow order, and together they define the transfer function -for the entire basic block. It's also possible to define an effect for +a basic block to define its own transfer function. +For brevity, these individual transfer functions are known as "effects". +Each effect is applied successively in dataflow order, +and together they define the transfer function for the entire basic block. +It's also possible to define an effect for particular outgoing edges of some terminators (e.g. -[`apply_call_return_effect`] for the `success` edge of a `Call` -terminator). Collectively, these are referred to as "per-edge effects". +[`apply_call_return_effect`] for the `success` edge of a `Call` terminator). +Collectively, these are referred to as "per-edge effects". ### "Before" Effects Observant readers of the documentation may notice that there are actually *two* possible effects for each statement and terminator, the "before" effect and the -unprefixed (or "primary") effect. The "before" effects are applied immediately +unprefixed (or "primary") effect. +The "before" effects are applied immediately before the unprefixed effect **regardless of the direction of the analysis**. In other words, a backward analysis will apply the "before" effect and then the "primary" effect when computing the transfer function for a basic block, just @@ -53,7 +54,8 @@ like a forward analysis. The vast majority of analyses should use only the unprefixed effects: Having multiple effects for each statement makes it difficult for consumers to know -where they should be looking. However, the "before" variants can be useful in +where they should be looking. +However, the "before" variants can be useful in some scenarios, such as when the effect of the right-hand side of an assignment statement must be considered separately from the left-hand side. @@ -61,9 +63,10 @@ statement must be considered separately from the left-hand side. Your analysis must converge to "fixpoint", otherwise it will run forever. Converging to fixpoint is just another way of saying "reaching equilibrium". -In order to reach equilibrium, your analysis must obey some laws. One of the -laws it must obey is that the bottom value[^bottom-purpose] joined with some -other value equals the second value. Or, as an equation: +In order to reach equilibrium, your analysis must obey some laws. +One of the laws it must obey is that the bottom value[^bottom-purpose] joined with some +other value equals the second value. +Or, as an equation: > *bottom* join *x* = *x* @@ -76,22 +79,23 @@ law state above ensures that once the dataflow state reaches top, it will no longer change (the fixpoint will be top). [^bottom-purpose]: The bottom value's primary purpose is as the initial dataflow - state. Each basic block's entry state is initialized to bottom before the - analysis starts. + state. + Each basic block's entry state is initialized to bottom before the analysis starts. ## A Brief Example -This section provides a brief example of a simple data-flow analysis at a high -level. It doesn't explain everything you need to know, but hopefully it will +This section provides a brief example of a simple data-flow analysis at a high level. +It doesn't explain everything you need to know, but hopefully it will make the rest of this page clearer. Let's say we want to do a simple analysis to find if `mem::transmute` may have -been called by a certain point in the program. Our analysis domain will just -be a `bool` that records whether `transmute` has been called so far. The bottom -value will be `false`, since by default `transmute` has not been called. The top -value will be `true`, since our analysis is done as soon as we determine that +been called by a certain point in the program. +Our analysis domain will just be a `bool` that records whether `transmute` has been called so far. +The bottom value will be `false`, since by default `transmute` has not been called. +The top value will be `true`, since our analysis is done as soon as we determine that `transmute` has been called. Our join operator will just be the boolean OR (`||`) -operator. We use OR and not AND because of this case: +operator. +We use OR and not AND because of this case: ```rust # unsafe fn example(some_cond: bool) { @@ -111,11 +115,11 @@ println!("x: {}", x); Once you have constructed an analysis, you must call `iterate_to_fixpoint` which will return a `Results`, which contains the dataflow state at fixpoint -upon entry of each block. Once you have a `Results`, you can inspect the -dataflow state at fixpoint at any point in the CFG. If you only need the state +upon entry of each block. +Once you have a `Results`, you can inspect the dataflow state at fixpoint at any point in the CFG. +If you only need the state at a few locations (e.g., each `Drop` terminator) use a [`ResultsCursor`]. If -you need the state at *every* location, a [`ResultsVisitor`] will be more -efficient. +you need the state at *every* location, a [`ResultsVisitor`] will be more efficient. ```text Analysis @@ -161,16 +165,16 @@ for (bb, block) in body.basic_blocks().iter_enumerated() { ### Graphviz Diagrams -When the results of a dataflow analysis are not what you expect, it often helps -to visualize them. This can be done with the `-Z dump-mir` flags described in -[Debugging MIR]. Start with `-Z dump-mir=F -Z dump-mir-dataflow`, where `F` is +When the results of a dataflow analysis are not what you expect, it often helps to visualize them. +This can be done with the `-Z dump-mir` flags described in [Debugging MIR]. +Start with `-Z dump-mir=F -Z dump-mir-dataflow`, where `F` is either "all" or the name of the MIR body you are interested in. These `.dot` files will be saved in your `mir_dump` directory and will have the [`NAME`] of the analysis (e.g. `maybe_inits`) as part of their filename. Each visualization will display the full dataflow state at entry and exit of each -block, as well as any changes that occur in each statement and terminator. See -the example below: +block, as well as any changes that occur in each statement and terminator. +See the example below: ![A graphviz diagram for a dataflow analysis](../img/dataflow-graphviz-example.png) @@ -189,4 +193,4 @@ the example below: [pr-1295]: https://github.com/rust-lang/rustc-dev-guide/pull/1295 [pr-1295-comment]: https://github.com/rust-lang/rustc-dev-guide/pull/1295#issuecomment-1118131294 [lattice]: https://en.wikipedia.org/wiki/Lattice_(order) -[wiki]: https://en.wikipedia.org/wiki/Data-flow_analysis#Basic_principles +[dataflow analysis]: https://en.wikipedia.org/wiki/Data-flow_analysis#Basic_principles diff --git a/src/doc/rustc-dev-guide/src/mir/debugging.md b/src/doc/rustc-dev-guide/src/mir/debugging.md index eec2737a423c5..7e1d5b4fa9da3 100644 --- a/src/doc/rustc-dev-guide/src/mir/debugging.md +++ b/src/doc/rustc-dev-guide/src/mir/debugging.md @@ -4,24 +4,24 @@ The `-Z dump-mir` flag can be used to dump a text representation of the MIR. The following optional flags, used in combination with `-Z dump-mir`, enable additional output formats, including: -* `-Z dump-mir-graphviz` - dumps a `.dot` file that represents MIR as a -control-flow graph +* `-Z dump-mir-graphviz` - dumps a `.dot` file that represents MIR as a control-flow graph * `-Z dump-mir-dataflow` - dumps a `.dot` file showing the [dataflow state] at each point in the control-flow graph `-Z dump-mir=F` is a handy compiler option that will let you view the MIR for -each function at each stage of compilation. `-Z dump-mir` takes a **filter** `F` -which allows you to control which functions and which passes you are -interested in. For example: +each function at each stage of compilation. +`-Z dump-mir` takes a **filter** `F` +which allows you to control which functions and which passes you are interested in. +For example: ```bash > rustc -Z dump-mir=foo ... ``` This will dump the MIR for any function whose name contains `foo`; it -will dump the MIR both before and after every pass. Those files will -be created in the `mir_dump` directory. There will likely be quite a -lot of them! +will dump the MIR both before and after every pass. +Those files will be created in the `mir_dump` directory. +There will likely be quite a lot of them! ```bash > cat > foo.rs @@ -34,8 +34,8 @@ fn main() { 161 ``` -The files have names like `rustc.main.000-000.CleanEndRegions.after.mir`. These -names have a number of parts: +The files have names like `rustc.main.000-000.CleanEndRegions.after.mir`. +These names have a number of parts: ```text rustc.main.000-000.CleanEndRegions.after.mir @@ -46,9 +46,9 @@ rustc.main.000-000.CleanEndRegions.after.mir def-path to the function etc being dumped ``` -You can also make more selective filters. For example, `main & CleanEndRegions` -will select for things that reference *both* `main` and the pass -`CleanEndRegions`: +You can also make more selective filters. +For example, `main & CleanEndRegions` +will select for things that reference *both* `main` and the pass `CleanEndRegions`: ```bash > rustc -Z dump-mir='main & CleanEndRegions' foo.rs @@ -58,8 +58,7 @@ rustc.main.000-000.CleanEndRegions.after.mir rustc.main.000-000.CleanEndRegions. Filters can also have `|` parts to combine multiple sets of `&`-filters. For example `main & CleanEndRegions | main & -NoLandingPads` will select *either* `main` and `CleanEndRegions` *or* -`main` and `NoLandingPads`: +NoLandingPads` will select *either* `main` and `CleanEndRegions` *or* `main` and `NoLandingPads`: ```bash > rustc -Z dump-mir='main & CleanEndRegions | main & NoLandingPads' foo.rs diff --git a/src/doc/rustc-dev-guide/src/mir/drop-elaboration.md b/src/doc/rustc-dev-guide/src/mir/drop-elaboration.md index 7ef60f4cca00b..7445ea2d980c1 100644 --- a/src/doc/rustc-dev-guide/src/mir/drop-elaboration.md +++ b/src/doc/rustc-dev-guide/src/mir/drop-elaboration.md @@ -2,7 +2,7 @@ ## Dynamic drops -According to the [reference][reference-drop]: +According to [Rust Reference][reference-drop]: > When an initialized variable or temporary goes out of scope, its destructor > is run, or it is dropped. Assignment also runs the destructor of its @@ -10,11 +10,12 @@ According to the [reference][reference-drop]: > initialized, only its initialized fields are dropped. When building the MIR, the `Drop` and `DropAndReplace` terminators represent -places where drops may occur. However, in this phase, the presence of these -terminators does not guarantee that a destructor will run. That's because the -target of a drop may be uninitialized (usually because it has been moved from) -before the terminator is reached. In general, we cannot know at compile-time whether a -variable is initialized. +places where drops may occur. +However, in this phase, the presence of these +terminators does not guarantee that a destructor will run. +That's because the target of a drop may be uninitialized (usually because it has been moved from) +before the terminator is reached. +In general, we cannot know at compile-time whether a variable is initialized. ```rust let mut y = vec![]; @@ -27,9 +28,8 @@ let mut y = vec![]; } // `x` goes out of scope here. Should it be dropped? ``` -In these cases, we need to keep track of whether a variable is initialized -*dynamically*. The rules are laid out in detail in [RFC 320: Non-zeroing -dynamic drops][RFC 320]. +In these cases, we need to keep track of whether a variable is initialized _dynamically_. +The rules are laid out in detail in [RFC 320: Non-zeroing dynamic drops][RFC 320]. ## Drop obligations @@ -46,17 +46,19 @@ From the RFC: When a structural path is moved from (and thus becomes uninitialized), any drop obligations for that path or its descendants (`path.f`, `path.f.g.h`, etc.) are -released. Types with `Drop` implementations do not permit moves from individual +released. +Types with `Drop` implementations do not permit moves from individual fields, so there is no need to track initializedness through them. When a local variable goes out of scope (`Drop`), or when a structural path is overwritten via assignment (`DropAndReplace`), we check for any drop -obligations for that variable or path. Unless that obligation has been +obligations for that variable or path. + Unless that obligation has been released by this point, its associated `Drop` implementation will be called. -For `enum` types, only fields corresponding to the "active" variant need to be -dropped. When processing drop obligations for such types, we first check the -discriminant to determine the active variant. All drop obligations for variants -besides the active one are ignored. +For `enum` types, only fields corresponding to the "active" variant need to be dropped. +When processing drop obligations for such types, we first check the +discriminant to determine the active variant. +All drop obligations for variants besides the active one are ignored. Here are a few interesting types to help illustrate these rules: @@ -80,89 +82,90 @@ enum MaybeDrop { ## Drop elaboration One valid model for these rules is to keep a boolean flag (a "drop flag") for -every structural path that is used at any point in the function. This flag is -set when its path is initialized and is cleared when the path is moved from. +every structural path that is used at any point in the function. +This flag is set when its path is initialized and is cleared when the path is moved from. When a `Drop` occurs, we check the flags for every obligation associated with -the target of the `Drop` and call the associated `Drop` impl for those that are -still applicable. +the target of the `Drop` and call the associated `Drop` impl for those that are still applicable. This process—transforming the newly built MIR with its imprecise `Drop` and -`DropAndReplace` terminators into one with drop flags—is known as drop -elaboration. When a MIR statement causes a variable to become initialized (or -uninitialized), drop elaboration inserts code that sets (or clears) the drop -flag for that variable. It wraps `Drop` terminators in conditionals that check -the newly inserted drop flags. +`DropAndReplace` terminators into one with drop flags—is known as drop elaboration. +When a MIR statement causes a variable to become initialized (or +uninitialized), drop elaboration inserts code that sets (or clears) the drop flag for that variable. +It wraps `Drop` terminators in conditionals that check the newly inserted drop flags. Drop elaboration also splits `DropAndReplace` terminators into a `Drop` of the -target and a write of the newly dropped place. This is somewhat unrelated to what -we've discussed above. +target and a write of the newly dropped place. +This is somewhat unrelated to what we've discussed above. Once this is complete, `Drop` terminators in the MIR correspond to a call to -the "drop glue" or "drop shim" for the type of the dropped place. The drop -glue for a type calls the `Drop` impl for that type (if one exists), and then +the "drop glue" or "drop shim" for the type of the dropped place. +The drop glue for a type calls the `Drop` impl for that type (if one exists), and then recursively calls the drop glue for all fields of that type. ## Drop elaboration in `rustc` -The approach described above is more expensive than necessary. One can imagine -a few optimizations: +The approach described above is more expensive than necessary. +One can imagine a few optimizations: -- Only paths that are the target of a `Drop` (or have the target as a prefix) - need drop flags. -- Some variables are known to be initialized (or uninitialized) when they are - dropped. These do not need drop flags. +- Only paths that are the target of a `Drop` (or have the target as a prefix) need drop flags. +- Some variables are known to be initialized (or uninitialized) when they are dropped. + These do not need drop flags. - If a set of paths are only dropped or moved from via a shared prefix, those paths can share a single drop flag. A subset of these are implemented in `rustc`. -In the compiler, drop elaboration is split across several modules. The pass -itself is defined [here][drops-transform], but the [main logic][drops] is +In the compiler, drop elaboration is split across several modules. +The pass itself is defined [here][drops-transform], but the [main logic][drops] is defined elsewhere since it is also used to build [drop shims][drops-shim]. -Drop elaboration designates each `Drop` in the newly built MIR as one of four -kinds: +Drop elaboration designates each `Drop` in the newly built MIR as one of four kinds: - `Static`, the target is always initialized. - `Dead`, the target is always **un**initialized. -- `Conditional`, the target is either wholly initialized or wholly - uninitialized. It is not partly initialized. +- `Conditional`, the target is either wholly initialized or wholly uninitialized. + It is not partly initialized. - `Open`, the target may be partly initialized. For this, it uses a pair of dataflow analyses, `MaybeInitializedPlaces` and -`MaybeUninitializedPlaces`. If a place is in one but not the other, then the +`MaybeUninitializedPlaces`. +If a place is in one but not the other, then the initializedness of the target is known at compile-time (`Dead` or `Static`). -In this case, drop elaboration does not add a flag for the target. It simply -removes (`Dead`) or preserves (`Static`) the `Drop` terminator. +In this case, drop elaboration does not add a flag for the target. +It simply removes (`Dead`) or preserves (`Static`) the `Drop` terminator. For `Conditional` drops, we know that the initializedness of the variable as a -whole is the same as the initializedness of its fields. Therefore, once we -generate a drop flag for the target of that drop, it's safe to call the drop +whole is the same as the initializedness of its fields. +Therefore, once we generate a drop flag for the target of that drop, it's safe to call the drop glue for that target. ### `Open` drops `Open` drops are the most complex, since we need to break down a single `Drop` terminator into several different ones, one for each field of the target whose -type has drop glue (`Ty::needs_drop`). We cannot call the drop glue for the +type has drop glue (`Ty::needs_drop`). +We cannot call the drop glue for the target itself because that requires all fields of the target to be initialized. Remember, variables whose type has a custom `Drop` impl do not allow `Open` drops because their fields cannot be moved from. This is accomplished by recursively categorizing each field as `Dead`, -`Static`, `Conditional` or `Open`. Fields whose type does not have drop glue -are automatically `Dead` and need not be considered during the recursion. When -we reach a field whose kind is not `Open`, we handle it as we did above. If the -field is also `Open`, the recursion continues. - -It's worth noting how we handle `Open` drops of enums. Inside drop elaboration, +`Static`, `Conditional`, or `Open`. +Fields whose type does not have drop glue +are automatically `Dead` and need not be considered during the recursion. +When we reach a field whose kind is not `Open`, we handle it as we did above. +If the field is also `Open`, the recursion continues. + +It's worth noting how we handle `Open` drops of enums. +Inside drop elaboration, each variant of the enum is treated like a field, with the invariant that only -one of those "variant fields" can be initialized at any given time. In the -general case, we do not know which variant is the active one, so we will have +one of those "variant fields" can be initialized at any given time. +In the general case, we do not know which variant is the active one, so we will have to call the drop glue for the enum (which checks the discriminant) or check the -discriminant ourselves as part of an elaborated `Open` drop. However, in -certain cases (within a `match` arm, for example) we do know which variant of -an enum is active. This information is encoded in the `MaybeInitializedPlaces` +discriminant ourselves as part of an elaborated `Open` drop. +However, in certain cases (within a `match` arm, for example) we do know which variant of +an enum is active. +This information is encoded in the `MaybeInitializedPlaces` and `MaybeUninitializedPlaces` dataflow analyses by marking all places corresponding to inactive variants as uninitialized. @@ -173,16 +176,18 @@ TODO: Discuss drop elaboration and unwinding. ## Aside: drop elaboration and const-eval In Rust, functions that are eligible for evaluation at compile-time must be -marked explicitly using the `const` keyword. This includes implementations of -the `Drop` trait, which may or may not be `const`. Code that is eligible for -compile-time evaluation may only call `const` functions, so any calls to +marked explicitly using the `const` keyword. +This includes implementations of the `Drop` trait, which may or may not be `const`. +Code that is eligible for compile-time evaluation may only call `const` functions, so any calls to non-const `Drop` implementations in such code must be forbidden. -A call to a `Drop` impl is encoded as a `Drop` terminator in the MIR. However, +A call to a `Drop` impl is encoded as a `Drop` terminator in the MIR. +However, as we discussed above, a `Drop` terminator in newly built MIR does not -necessarily result in a call to `Drop::drop`. The drop target may be -uninitialized at that point. This means that checking for non-const `Drop`s on -the newly built MIR can result in spurious errors. Instead, we wait until after +necessarily result in a call to `Drop::drop`. +The drop target may be uninitialized at that point. +This means that checking for non-const `Drop`s on the newly built MIR can result in spurious errors. +Instead, we wait until after drop elaboration runs, which eliminates `Dead` drops (ones where the target is known to be uninitialized) to run these checks. diff --git a/src/doc/rustc-dev-guide/src/mir/index.md b/src/doc/rustc-dev-guide/src/mir/index.md index 8ba5f3ac8b784..45d72226807a7 100644 --- a/src/doc/rustc-dev-guide/src/mir/index.md +++ b/src/doc/rustc-dev-guide/src/mir/index.md @@ -1,15 +1,15 @@ # The MIR (Mid-level IR) -MIR is Rust's _Mid-level Intermediate Representation_. It is -constructed from [HIR](../hir.html). MIR was introduced in -[RFC 1211]. It is a radically simplified form of Rust that is used for -certain flow-sensitive safety checks – notably the borrow checker! – -and also for optimization and code generation. +MIR is Rust's _Mid-level Intermediate Representation_. +It is constructed from [HIR](../hir.html). +MIR was introduced in [RFC 1211]. +It is a radically simplified form of Rust that is used for +certain flow-sensitive safety checks – notably the borrow checker! +– and also for optimization and code generation. If you'd like a very high-level introduction to MIR, as well as some of the compiler concepts that it relies on (such as control-flow -graphs and desugaring), you may enjoy the -[rust-lang blog post that introduced MIR][blog]. +graphs and desugaring), you may enjoy the [rust-lang blog post that introduced MIR][blog]. [blog]: https://blog.rust-lang.org/2016/04/19/MIR.html @@ -36,28 +36,24 @@ This section introduces the key concepts of MIR, summarized here: - **Basic blocks**: units of the control-flow graph, consisting of: - **statements:** actions with one successor - - **terminators:** actions with potentially multiple successors; always at - the end of a block - - (if you're not familiar with the term *basic block*, see the [background - chapter][cfg]) + - **terminators:** actions with potentially multiple successors; always at the end of a block + - (if you're not familiar with the term *basic block*, see the [background chapter][cfg]) - **Locals:** Memory locations allocated on the stack (conceptually, at - least), such as function arguments, local variables, and - temporaries. These are identified by an index, written with a - leading underscore, like `_1`. There is also a special "local" - (`_0`) allocated to store the return value. -- **Places:** expressions that identify a location in memory, like `_1` or - `_1.f`. -- **Rvalues:** expressions that produce a value. The "R" stands for - the fact that these are the "right-hand side" of an assignment. + least), such as function arguments, local variables, and temporaries. + These are identified by an index, written with a leading underscore, like `_1`. + There is also a special "local" (`_0`) allocated to store the return value. +- **Places:** expressions that identify a location in memory, like `_1` or `_1.f`. +- **Rvalues:** expressions that produce a value. + The "R" stands for the fact that these are the "right-hand side" of an assignment. - **Operands:** the arguments to an rvalue, which can either be a constant (like `22`) or a place (like `_1`). You can get a feeling for how MIR is constructed by translating simple -programs into MIR and reading the pretty printed output. In fact, the -playground makes this easy, since it supplies a MIR button that will -show you the MIR for your program. Try putting this program into play -(or [clicking on this link][sample-play]), and then clicking the "MIR" -button on the top: +programs into MIR and reading the pretty printed output. +In fact, the playground makes this easy, since it supplies a MIR button that will +show you the MIR for your program. +Try putting this program into play +(or [clicking on this link][sample-play]), and then clicking the "MIR" button on the top: [sample-play]: https://play.rust-lang.org/?gist=30074856e62e74e91f06abd19bd72ece&version=stable&edition=2021 @@ -88,7 +84,8 @@ This requires the nightly toolchain. **Variable declarations.** If we drill in a bit, we'll see it begins -with a bunch of variable declarations. They look like this: +with a bunch of variable declarations. +They look like this: ```mir let mut _0: (); // return place @@ -124,8 +121,7 @@ annotated with `// in scope 0` would be missing `vec`, if you were stepping through the code in a debugger, for example. **Basic blocks.** Reading further, we see our first **basic block** (naturally -it may look slightly different when you view it, and I am ignoring some of the -comments): +it may look slightly different when you view it, and I am ignoring some of the comments): ```mir bb0: { @@ -143,9 +139,8 @@ StorageLive(_1); This statement indicates that the variable `_1` is "live", meaning that it may be used later – this will persist until we encounter a -`StorageDead(_1)` statement, which indicates that the variable `_1` is -done being used. These "storage statements" are used by LLVM to -allocate stack space. +`StorageDead(_1)` statement, which indicates that the variable `_1` is done being used. +These "storage statements" are used by LLVM to allocate stack space. The **terminator** of the block `bb0` is the call to `Vec::new`: @@ -154,8 +149,8 @@ _1 = const >::new() -> bb2; ``` Terminators are different from statements because they can have more -than one successor – that is, control may flow to different -places. Function calls like the call to `Vec::new` are always +than one successor – that is, control may flow to different places. +Function calls like the call to `Vec::new` are always terminators because of the possibility of unwinding, although in the case of `Vec::new` we are able to see that indeed unwinding is not possible, and hence we list only one successor block, `bb2`. @@ -183,11 +178,10 @@ Assignments in general have the form: = ``` -A place is an expression like `_3`, `_3.f` or `*_3` – it denotes a -location in memory. An **Rvalue** is an expression that creates a -value: in this case, the rvalue is a mutable borrow expression, which -looks like `&mut `. So we can kind of define a grammar for -rvalues like so: +A place is an expression like `_3`, `_3.f` or `*_3` – it denotes a location in memory. +An **Rvalue** is an expression that creates a +value: in this case, the rvalue is a mutable borrow expression, which looks like `&mut `. +So we can kind of define a grammar for rvalues like so: ```text = & (mut)? @@ -201,12 +195,12 @@ rvalues like so: ``` As you can see from this grammar, rvalues cannot be nested – they can -only reference places and constants. Moreover, when you use a place, +only reference places and constants. +Moreover, when you use a place, we indicate whether we are **copying it** (which requires that the -place have a type `T` where `T: Copy`) or **moving it** (which works -for a place of any type). So, for example, if we had the expression `x -= a + b + c` in Rust, that would get compiled to two statements and a -temporary: +place have a type `T` where `T: Copy`) or **moving it** (which works for a place of any type). +So, for example, if we had the expression `x += a + b + c` in Rust, that would get compiled to two statements and a temporary: ```mir TMP1 = a + b @@ -220,33 +214,33 @@ over the overflow checks.) ## MIR data types -The MIR data types are defined in the [`compiler/rustc_middle/src/mir/`][mir] -module. Each of the key concepts mentioned in the previous section +The MIR data types are defined in the [`compiler/rustc_middle/src/mir/`][mir] module. +Each of the key concepts mentioned in the previous section maps in a fairly straightforward way to a Rust type. -The main MIR data type is [`Body`]. It contains the data for a single +The main MIR data type is [`Body`]. +It contains the data for a single function (along with sub-instances of Mir for "promoted constants", -but [you can read about those below](#promoted)). +but [you can read about those below](#promoted-constants)). - **Basic blocks**: The basic blocks are stored in the field - [`Body::basic_blocks`][basicblocks]; this is a vector - of [`BasicBlockData`] structures. Nobody ever references a - basic block directly: instead, we pass around [`BasicBlock`] + [`Body::basic_blocks`][basicblocks]; this is a vector of [`BasicBlockData`] structures. + Nobody ever references a basic block directly: instead, we pass around [`BasicBlock`] values, which are [newtype'd] indices into this vector. - **Statements** are represented by the type [`Statement`]. - **Terminators** are represented by the [`Terminator`]. - **Locals** are represented by a [newtype'd] index type [`Local`]. - The data for a local variable is found in the - [`Body::local_decls`][localdecls] vector. There is also a special constant + The data for a local variable is found in the [`Body::local_decls`][localdecls] vector. + There is also a special constant [`RETURN_PLACE`] identifying the special "local" representing the return value. -- **Places** are identified by the struct [`Place`]. There are a few - fields: +- **Places** are identified by the struct [`Place`]. + There are a few fields: - Local variables like `_1` - - **Projections**, which are fields or other things that "project - out" from a base place. These are represented by the [newtype'd] type + - **Projections**, which are fields or other things that "project out" from a base place. + These are represented by the [newtype'd] type [`ProjectionElem`]. So e.g. the place `_1.f` is a projection, - with `f` being the "projection element" and `_1` being the base - path. `*_1` is also a projection, with the `*` being represented + with `f` being the "projection element" and `_1` being the base path. + `*_1` is also a projection, with the `*` being represented by the [`ProjectionElem::Deref`] element. - **Rvalues** are represented by the enum [`Rvalue`]. - **Operands** are represented by the enum [`Operand`]. @@ -256,12 +250,14 @@ but [you can read about those below](#promoted)). When code has reached the MIR stage, constants can generally come in two forms: *MIR constants* ([`mir::Constant`]) and *type system constants* ([`ty::Const`]). MIR constants are used as operands: in `x + CONST`, `CONST` is a MIR constant; -similarly, in `x + 2`, `2` is a MIR constant. Type system constants are used in +similarly, in `x + 2`, `2` is a MIR constant. +Type system constants are used in the type system, in particular for array lengths but also for const generics. Generally, both kinds of constants can be "unevaluated" or "already evaluated". An unevaluated constant simply stores the `DefId` of what needs to be evaluated -to compute this result. An evaluated constant (a "value") has already been +to compute this result. +An evaluated constant (a "value") has already been computed; their representation differs between type system constants and MIR constants: MIR constants evaluate to a `mir::ConstValue`; type system constants evaluate to a `ty::ValTree`. @@ -275,22 +271,24 @@ parameter is used as an operand. ### MIR constant values In general, a MIR constant value (`mir::ConstValue`) was computed by evaluating -some constant the user wrote. This [const evaluation](../const-eval.md) produces -a very low-level representation of the result in terms of individual bytes. We -call this an "indirect" constant (`mir::ConstValue::Indirect`) since the value +some constant the user wrote. +This [const evaluation](../const-eval.md) produces +a very low-level representation of the result in terms of individual bytes. +We call this an "indirect" constant (`mir::ConstValue::Indirect`) since the value is stored in-memory. -However, storing everything in-memory would be awfully inefficient. Hence there -are some other variants in `mir::ConstValue` that can represent certain simple -and common values more efficiently. In particular, everything that can be +However, storing everything in-memory would be awfully inefficient. +Hence there are some other variants in `mir::ConstValue` that can represent certain simple +and common values more efficiently. +In particular, everything that can be directly written as a literal in Rust (integers, floats, chars, bools, but also `"string literals"` and `b"byte string literals"`) has an optimized variant that avoids the full overhead of the in-memory representation. ### ValTrees -An evaluated type system constant is a "valtree". The `ty::ValTree` datastructure -allows us to represent +An evaluated type system constant is a "valtree". +The `ty::ValTree` datastructure allows us to represent * arrays, * many structs, @@ -298,32 +296,33 @@ allows us to represent * enums and, * most primitives. -The most important rule for -this representation is that every value must be uniquely represented. In other -words: a specific value must only be representable in one specific way. For example: there is only -one way to represent an array of two integers as a `ValTree`: +The most important rule for this representation is that every value must be uniquely represented. +In other words, a specific value must only be representable in one specific way. +For example, there is only one way to represent an array of two integers as a `ValTree`: `Branch([Leaf(first_int), Leaf(second_int)])`. Even though theoretically a `[u32; 2]` could be encoded in a `u64` and thus just be a `Leaf(bits_of_two_u32)`, that is not a legal construction of `ValTree` (and is very complex to do, so it is unlikely anyone is tempted to do so). -These rules also mean that some values are not representable. There can be no `union`s in type -level constants, as it is not clear how they should be represented, because their active variant -is unknown. Similarly there is no way to represent raw pointers, as addresses are unknown at -compile-time and thus we cannot make any assumptions about them. References on the other hand -*can* be represented, as equality for references is defined as equality on their value, so we -ignore their address and just look at the backing value. We must make sure that the pointer values -of the references are not observable at compile time. We thus encode `&42` exactly like `42`. -Any conversion from -valtree back to a MIR constant value must reintroduce an actual indirection. At codegen time the -addresses may be deduplicated between multiple uses or not, entirely depending on arbitrary -optimization choices. +These rules also mean that some values are not representable. +There can be no `union`s in type level constants, +as it is not clear how they should be represented, +because their active variant is unknown. +Similarly, there is no way to represent raw pointers as addresses are unknown at compile-time, +and thus we cannot make any assumptions about them. +References on the other hand *can* be represented, +as equality for references is defined as equality on their value, +so we ignore their address and just look at the backing value. +We must make sure that the pointer values of the references are not observable at compile time. +We thus encode `&42` exactly like `42`. +Any conversion from valtree back to a MIR constant value must reintroduce an actual indirection. +At codegen time, +the addresses may be deduplicated between multiple uses or not, +entirely depending on arbitrary optimization choices. As a consequence, all decoding of `ValTree` must happen by matching on the type first and making -decisions depending on that. The value itself gives no useful information without the type that -belongs to it. - - +decisions depending on that. +The value itself gives no useful information without the type that belongs to it. ### Promoted constants diff --git a/src/doc/rustc-dev-guide/src/mir/optimizations.md b/src/doc/rustc-dev-guide/src/mir/optimizations.md index 964fa063a680f..237177ba56c0d 100644 --- a/src/doc/rustc-dev-guide/src/mir/optimizations.md +++ b/src/doc/rustc-dev-guide/src/mir/optimizations.md @@ -1,44 +1,47 @@ # MIR optimizations -MIR optimizations are optimizations run on the [MIR][mir] to produce better MIR -before codegen. This is important for two reasons: first, it makes the final +MIR optimizations are optimizations run on the [MIR] to produce better MIR before codegen. +This is important for two reasons: first, it makes the final generated executable code better, and second, it means that LLVM has less work -to do, so compilation is faster. Note that since MIR is generic (not -[monomorphized][monomorph] yet), these optimizations are particularly -effective; we can optimize the generic version, so all of the monomorphizations -are cheaper! +to do, so compilation is faster. +Note that since MIR is generic (not +[monomorphized] yet), these optimizations are particularly +effective; we can optimize the generic version, so all of the monomorphizations are cheaper! [mir]: ../mir/index.md -[monomorph]: ../appendix/glossary.md#mono +[monomorphized]: ../appendix/glossary.md#mono -MIR optimizations run after borrow checking. We run a series of optimization -passes over the MIR to improve it. Some passes are required to run on all code, +MIR optimizations run after borrow checking. +We run a series of optimization passes over the MIR to improve it. +Some passes are required to run on all code, some passes don't actually do optimizations but only check stuff, and some passes are only turned on in `release` mode. -The [`optimized_mir`][optmir] [query] is called to produce the optimized MIR -for a given [`DefId`][defid]. This query makes sure that the borrow checker has -run and that some validation has occurred. Then, it [steals][steal] the MIR, +The [`optimized_mir`] [query] is called to produce the optimized MIR +for a given [`DefId`]. +This query makes sure that the borrow checker has run and that some validation has occurred. +Then, it [steals] the MIR, optimizes it, and returns the improved MIR. -[optmir]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir_transform/fn.optimized_mir.html +[`optimized_mir`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir_transform/fn.optimized_mir.html [query]: ../query.md -[defid]: ../appendix/glossary.md#def-id -[steal]: ../mir/passes.md#stealing +[`defid`]: ../appendix/glossary.md#def-id +[steals]: ../mir/passes.md#stealing ## Quickstart for adding a new optimization -1. Make a Rust source file in `tests/mir-opt` that shows the code you want to - optimize. This should be kept simple, so avoid `println!` or other formatting - code if it's not necessary for the optimization. The reason for this is that +1. Make a Rust source file in `tests/mir-opt` that shows the code you want to optimize. + This should be kept simple, so avoid `println!` or other formatting + code if it's not necessary for the optimization. + The reason for this is that `println!`, `format!`, etc. generate a lot of MIR that can make it harder to understand what the optimization does to the test. -2. Run `./x test --bless tests/mir-opt/.rs` to generate a MIR - dump. Read [this README][mir-opt-test-readme] for instructions on how to dump - things. +2. Run `./x test --bless tests/mir-opt/.rs` to generate a MIR dump. + Read [this README][mir-opt-test-readme] for instructions on how to dump things. -3. Commit the current working directory state. The reason you should commit the +3. Commit the current working directory state. + The reason you should commit the test output before you implement the optimization is so that you (and your reviewers) can see a before/after diff of what the optimization changed. @@ -47,28 +50,26 @@ optimizes it, and returns the improved MIR. 1. pick a small optimization (such as [`remove_storage_markers`]) and copy it to a new file, - 2. add your optimization to one of the lists in the - [`run_optimization_passes()`] function, + 2. add your optimization to one of the lists in the [`run_optimization_passes()`] function, 3. and then start modifying the copied optimization. -5. Rerun `./x test --bless tests/mir-opt/.rs` to regenerate the - MIR dumps. Look at the diffs to see if they are what you expect. +5. Rerun `./x test --bless tests/mir-opt/.rs` to regenerate the MIR dumps. + Look at the diffs to see if they are what you expect. 6. Run `./x test tests/ui` to see if your optimization broke anything. -7. If there are issues with your optimization, experiment with it a bit and - repeat steps 5 and 6. +7. If there are issues with your optimization, experiment with it a bit and repeat steps 5 and 6. -8. Commit and open a PR. You can do this at any point, even if things aren't - working yet, so that you can ask for feedback on the PR. Open a "WIP" PR - (just prefix your PR title with `[WIP]` or otherwise note that it is a +8. Commit and open a PR. + You can do this at any point, even if things aren't + working yet, so that you can ask for feedback on the PR. + Open a "WIP" PR (just prefix your PR title with `[WIP]` or otherwise note that it is a work in progress) in that case. - Make sure to commit the blessed test output as well! It's necessary for CI to - pass and it's very helpful to reviewers. + Make sure to commit the blessed test output as well! + It's necessary for CI to pass and it's very helpful to reviewers. -If you have any questions along the way, feel free to ask in -`#t-compiler/wg-mir-opt` on Zulip. +If you have any questions along the way, feel free to ask in `#t-compiler/wg-mir-opt` on Zulip. [mir-opt-test-readme]: https://github.com/rust-lang/rust/blob/HEAD/tests/mir-opt/README.md [`compiler/rustc_mir_transform/src`]: https://github.com/rust-lang/rust/tree/HEAD/compiler/rustc_mir_transform/src @@ -78,10 +79,11 @@ If you have any questions along the way, feel free to ask in ## Defining optimization passes The list of passes run and the order in which they are run is defined by the -[`run_optimization_passes`][rop] function. It contains an array of passes to -run. Each pass in the array is a struct that implements the [`MirPass`] trait. -The array is an array of `&dyn MirPass` trait objects. Typically, a pass is -implemented in its own module of the [`rustc_mir_transform`][trans] crate. +[`run_optimization_passes`][rop] function. +It contains an array of passes to run. +Each pass in the array is a struct that implements the [`MirPass`] trait. +The array is an array of `&dyn MirPass` trait objects. +Typically, a pass is implemented in its own module of the [`rustc_mir_transform`][trans] crate. [rop]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir_transform/fn.run_optimization_passes.html [`MirPass`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir_transform/pass_manager/trait.MirPass.html @@ -99,12 +101,14 @@ You can see the ["Implementors" section of the `MirPass` rustdocs][impl] for mor ## MIR optimization levels -MIR optimizations can come in various levels of readiness. Experimental -optimizations may cause miscompilations, or slow down compile times. +MIR optimizations can come in various levels of readiness. +Experimental optimizations may cause miscompilations, or slow down compile times. These passes are still included in nightly builds to gather feedback and make it easier to modify -the pass. To enable working with slow or otherwise experimental optimization passes, -you can specify the `-Z mir-opt-level` debug flag. You can find the -definitions of the levels in the [compiler MCP]. If you are developing a MIR pass and +the pass. +To enable working with slow or otherwise experimental optimization passes, +you can specify the `-Z mir-opt-level` debug flag. +You can find the definitions of the levels in the [compiler MCP]. +If you are developing a MIR pass and want to query whether your optimization pass should run, you can check the current level using [`tcx.sess.opts.unstable_opts.mir_opt_level`][mir_opt_level]. diff --git a/src/doc/rustc-dev-guide/src/mir/passes.md b/src/doc/rustc-dev-guide/src/mir/passes.md index 5e3dbb5984ba9..2ee7a4df5c9f4 100644 --- a/src/doc/rustc-dev-guide/src/mir/passes.md +++ b/src/doc/rustc-dev-guide/src/mir/passes.md @@ -5,16 +5,17 @@ If you would like to get the MIR: - for a function - you can use the `optimized_mir` query (typically used by codegen) or the `mir_for_ctfe` query (typically used by compile time function evaluation, i.e., *CTFE*); - for a promoted - you can use the `promoted_mir` query. -These will give you back the final, optimized MIR. For foreign def-ids, we simply read the MIR -from the other crate's metadata. But for local def-ids, the query will +These will give you back the final, optimized MIR. +For foreign def-ids, we simply read the MIR from the other crate's metadata. +But for local def-ids, the query will construct the optimized MIR by requesting a pipeline of upstream queries[^query]. Each query will contain a series of passes. This section describes how those queries and passes work and how you can extend them. To produce the optimized MIR for a given def-id `D`, `optimized_mir(D)` -goes through several suites of passes, each grouped by a -query. Each suite consists of passes which perform linting, analysis, transformation or -optimization. Each query represent a useful intermediate point +goes through several suites of passes, each grouped by a query. +Each suite consists of passes which perform linting, analysis, transformation, or optimization. +Each query represent a useful intermediate point where we can access the MIR dialect for type checking or other purposes: - `mir_built(D)` – it gives the initial MIR just after it's built; @@ -40,23 +41,23 @@ are defined in the [`rustc_mir_transform`][mirtransform] crate, the `MirPass` tr The MIR is therefore modified in place (which helps to keep things efficient). A basic example of a MIR pass is [`RemoveStorageMarkers`], which walks -the MIR and removes all storage marks if they won't be emitted during codegen. As you -can see from its source, a MIR pass is defined by first defining a -dummy type, a struct with no fields: +the MIR and removes all storage marks if they won't be emitted during codegen. +As you can see from its source, +a MIR pass is defined by first defining a dummy type, a struct with no fields: ```rust pub struct RemoveStorageMarkers; ``` -for which we implement the `MirPass` trait. We can then insert -this pass into the appropriate list of passes found in a query like +for which we implement the `MirPass` trait. +We can then insert this pass into the appropriate list of passes found in a query like `mir_built`, `optimized_mir`, etc. (If this is an optimization, it should go into the `optimized_mir` list.) Another example of a simple MIR pass is [`CleanupPostBorrowck`][cleanup-pass], which walks -the MIR and removes all statements that are not relevant to code generation. As you can see from -its [source][cleanup-source], it is defined by first defining a dummy type, a struct with no -fields: +the MIR and removes all statements that are not relevant to code generation. +As you can see from +its [source][cleanup-source], it is defined by first defining a dummy type, a struct with no fields: ```rust pub struct CleanupPostBorrowck; @@ -75,30 +76,28 @@ impl<'tcx> MirPass<'tcx> for CleanupPostBorrowck { We [register][pass-register] this pass inside the `mir_drops_elaborated_and_const_checked` query. (If this is an optimization, it should go into the `optimized_mir` list.) -If you are writing a pass, there's a good chance that you are going to -want to use a [MIR visitor]. MIR visitors are a handy way to walk all -the parts of the MIR, either to search for something or to make small -edits. +If you are writing a pass, there's a good chance that you are going to want to use a [MIR visitor]. +MIR visitors are a handy way to walk all +the parts of the MIR, either to search for something or to make small edits. ## Stealing The intermediate queries `mir_const()` and `mir_promoted()` yield up a `&'tcx Steal>`, allocated using `tcx.alloc_steal_mir()`. This indicates that the result may be **stolen** by a subsequent query – this is an -optimization to avoid cloning the MIR. Attempting to use a stolen -result will cause a panic in the compiler. Therefore, it is important -that you do not accidentally read from these intermediate queries without +optimization to avoid cloning the MIR. +Attempting to use a stolen result will cause a panic in the compiler. +Therefore, it is important that you do not accidentally read from these intermediate queries without the consideration of the dependency in the MIR processing pipeline. Because of this stealing mechanism, some care must be taken to ensure that, before the MIR at a particular phase in the processing -pipeline is stolen, anyone who may want to read from it has already -done so. +pipeline is stolen, anyone who may want to read from it has already done so. Concretely, this means that if you have a query `foo(D)` that wants to access the result of `mir_promoted(D)`, you need to have `foo(D)` -calling the `mir_const(D)` query first. This will force it -to execute even though you don't directly require its result. +calling the `mir_const(D)` query first. +This will force it to execute even though you don't directly require its result. > This mechanism is a bit dodgy. There is a discussion of more elegant alternatives in [rust-lang/rust#41710]. @@ -138,14 +137,16 @@ flowchart BT The stadium-shape queries (e.g., `mir_built`) with a deep color are the primary queries in the pipeline, while the rectangle-shape queries (e.g., `mir_const_qualif*`[^star]) with a shallow color -are those subsequent queries that need to read the results from `&'tcx Steal>`. With the -stealing mechanism, the rectangle-shape queries must be performed before any stadium-shape queries, +are those subsequent queries that need to read the results from `&'tcx Steal>`. +With the stealing mechanism, +the rectangle-shape queries must be performed before any stadium-shape queries, that have an equal or larger height in the dependency tree, ever do. [^part]: The `mir_promoted` query will yield up a tuple `(&'tcx Steal>, &'tcx Steal>>)`, `promoted_mir` will steal part 1 (`&'tcx Steal>>`) and `mir_drops_elaborated_and_const_checked` -will steal part 0 (`&'tcx Steal>`). And their stealing is irrelevant to each other, +will steal part 0 (`&'tcx Steal>`). +And their stealing is irrelevant to each other, i.e., can be performed separately. [^star]: Note that the `*` suffix in the queries represent a set of queries with the same prefix. @@ -154,11 +155,13 @@ For example, `mir_borrowck*` represents `mir_borrowck`, `mir_borrowck_const_arg` ### Example -As an example, consider MIR const qualification. It wants to read the result produced by the -`mir_const` query. However, that result will be **stolen** by the `mir_promoted` query at some -time in the pipeline. Before `mir_promoted` is ever queried, calling the `mir_const_qualif` query +As an example, consider MIR const qualification. +It wants to read the result produced by the `mir_const` query. +However, that result will be **stolen** by the `mir_promoted` query at some time in the pipeline. +Before `mir_promoted` is ever queried, calling the `mir_const_qualif` query will succeed since `mir_const` will produce (if queried the first time) or cache (if queried -multiple times) the `Steal` result and the result is **not** stolen yet. After `mir_promoted` is +multiple times) the `Steal` result and the result is **not** stolen yet. +After `mir_promoted` is queried, the result would be stolen and calling the `mir_const_qualif` query to read the result would cause a panic. diff --git a/src/doc/rustc-dev-guide/src/mir/visitor.md b/src/doc/rustc-dev-guide/src/mir/visitor.md index b0facee69a5fa..1b1797b993d4c 100644 --- a/src/doc/rustc-dev-guide/src/mir/visitor.md +++ b/src/doc/rustc-dev-guide/src/mir/visitor.md @@ -1,17 +1,16 @@ # MIR visitor The MIR visitor is a convenient tool for traversing the MIR and either -looking for things or making changes to it. The visitor traits are -defined in [the `rustc_middle::mir::visit` module][m-v] – there are two of +looking for things or making changes to it. +The visitor traits are defined in [the `rustc_middle::mir::visit` module][m-v] – there are two of them, generated via a single macro: `Visitor` (which operates on a `&Mir` and gives back shared references) and `MutVisitor` (which operates on a `&mut Mir` and gives back mutable references). [m-v]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/mir/visit/index.html -To implement a visitor, you have to create a type that represents -your visitor. Typically, this type wants to "hang on" to whatever -state you will need while processing MIR: +To implement a visitor, you have to create a type that represents your visitor. +Typically, this type wants to "hang on" to whatever state you will need while processing MIR: ```rust,ignore struct MyVisitor<...> { @@ -33,9 +32,10 @@ impl<'tcx> MutVisitor<'tcx> for MyVisitor { As shown above, within the impl, you can override any of the `visit_foo` methods (e.g., `visit_terminator`) in order to write some -code that will execute whenever a `foo` is found. If you want to -recursively walk the contents of the `foo`, you then invoke the -`super_foo` method. (NB. You never want to override `super_foo`.) +code that will execute whenever a `foo` is found. +If you want to recursively walk the contents of the `foo`, you then invoke the +`super_foo` method. +Note that you never want to override `super_foo`. A very simple example of a visitor can be found in [`LocalFinder`]. By implementing `visit_local` method, this visitor identifies local variables that @@ -52,4 +52,3 @@ post-order, and so forth). [t]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/mir/traversal/index.html [traversal]: https://en.wikipedia.org/wiki/Tree_traversal - diff --git a/src/doc/rustc-dev-guide/src/rustbot.md b/src/doc/rustc-dev-guide/src/rustbot.md index 88b831697ffa1..11a2404c2aa6e 100644 --- a/src/doc/rustc-dev-guide/src/rustbot.md +++ b/src/doc/rustc-dev-guide/src/rustbot.md @@ -2,8 +2,8 @@ `@rustbot` (also known as `triagebot`) is a utility robot that is mostly used to allow any contributor to achieve certain tasks that would normally require GitHub -membership to the `rust-lang` organization. Its most interesting features for -contributors to `rustc` are issue claiming and relabeling. +membership to the `rust-lang` organization. +Its most interesting features for contributors to `rustc` are issue claiming and relabeling. ## Issue claiming @@ -24,29 +24,31 @@ If you want to unassign from an issue, `@rustbot` has a different command: ## Issue relabeling -Changing labels for an issue or PR is also normally reserved for members of the -organization. However, `@rustbot` allows you to relabel an issue yourself, only -with a few restrictions. This is mostly useful in two cases: +Changing labels for an issue or PR is also normally reserved for members of the organization. +However, `@rustbot` allows you to relabel an issue yourself, only with a few restrictions. +This is mostly useful in two cases: **Helping with issue triage**: Rust's issue tracker has more than 5,000 open issues at the time of this writing, so labels are the most powerful tool that we -have to keep it as tidy as possible. You don't need to spend hours in the issue tracker +have to keep it as tidy as possible. +You don't need to spend hours in the issue tracker to triage issues, but if you open an issue, you should feel free to label it if you are comfortable with doing it yourself. -**Updating the status of a PR**: We use "status labels" to reflect the status of -PRs. For example, if your PR has merge conflicts, it will automatically be assigned -the `S-waiting-on-author`, and reviewers might not review it until you rebase your -PR. Once you do rebase your branch, you should change the labels yourself to remove -the `S-waiting-on-author` label and add back `S-waiting-on-review`. In this case, +**Updating the status of a PR**: We use "status labels" to reflect the status of PRs. +For example, if your PR has merge conflicts, it will automatically be assigned +the `S-waiting-on-author`, and reviewers might not review it until you rebase your PR. +Once you do rebase your branch, you should change the labels yourself to remove +the `S-waiting-on-author` label and add back `S-waiting-on-review`. +In this case, the `@rustbot` command will look like this: @rustbot label -S-waiting-on-author +S-waiting-on-review -The syntax for this command is pretty loose, so there are other variants of this -command invocation. There are also some shortcuts to update labels, +The syntax for this command is pretty loose, so there are other variants of this command invocation. +There are also some shortcuts to update labels, for instance `@rustbot ready` will do the same thing with above command. -For more details, see [the docs page about labeling][labeling] and [shortcuts][shortcuts]. +For more details, see [the docs page about labeling][labeling] and [shortcuts]. [labeling]: https://forge.rust-lang.org/triagebot/labeling.html [shortcuts]: https://forge.rust-lang.org/triagebot/shortcuts.html @@ -57,10 +59,11 @@ If you are interested in seeing what `@rustbot` is capable of, check out its [do which is meant as a reference for the bot and should be kept up to date every time the bot gets an upgrade. -`@rustbot` is maintained by the Release team. If you have any feedback regarding +`@rustbot` is maintained by the Release team. +If you have any feedback regarding existing commands or suggestions for new commands, feel free to reach out -[on Zulip][zulip] or file an issue in [the triagebot repository][repo] +[on Zulip] or file an issue in [the triagebot repository]. [documentation]: https://forge.rust-lang.org/triagebot/index.html -[zulip]: https://rust-lang.zulipchat.com/#narrow/stream/224082-t-release.2Ftriagebot -[repo]: https://github.com/rust-lang/triagebot/ +[on zulip]: https://rust-lang.zulipchat.com/#narrow/stream/224082-t-release.2Ftriagebot +[the triagebot repository]: https://github.com/rust-lang/triagebot/