Resolver: Parallelize the import resolution loop by LorrensP-2158466 · Pull Request #158845 · rust-lang/rust

LorrensP-2158466 · 2026-07-06T08:22:15Z

View all comments

Parallelize the import resolution loop using par_filter_map (introduced in this pr).

With the accompanying changes to make datastructures concurrent:

CmRefCell is now CmRwLock because of the borrow counters (nothing changes in single-threaded mode)
Cache(Ref)Cells are now changed into atomics/(rw)locks
populating external resolution tables now use Once to ensure only 1 thread builds the table and all others wait if they need it

This is the bare minimum to make the parallel loop work and is not at all optimized, this will follow :).

r? @petrochenkov

LorrensP-2158466 · 2026-07-06T08:22:55Z

Lets see what CI says.

I have tried to add comments to changes to show my thought process behind them, but i'll make another pass here as well to be sure.

petrochenkov · 2026-07-06T16:24:33Z

@bors try @rust-timer queue

…r=<try> Resolver: Parallelize the import resolution loop

rust-bors · 2026-07-06T18:42:55Z

☀️ Try build successful (CI)
Build commit: 3df1fed (3df1fed3aaf1081aa59cfe2228eed289a2ef744e)
Base parent: 3c00c96 (3c00c96d3af4d5b5e101e56cc161a608b21366ee)

…r=<try> Resolver: Parallelize the import resolution loop

rust-bors · 2026-07-15T14:05:54Z

☀️ Try build successful (CI)
Build commit: ba4d109 (ba4d109e8749aa102f90e5254b1aeb482456cfbe)
Base parent: 47101ad (47101adcea71daee3c2879218f5b883bcdf180aa)

rust-timer · 2026-07-15T14:55:30Z

Finished benchmarking commit (ba4d109): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.4%	[0.2%, 0.6%]	12
Regressions ❌ (secondary)	0.2%	[0.2%, 0.3%]	6
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.7%	[-0.8%, -0.7%]	6
All ❌✅ (primary)	0.4%	[0.2%, 0.6%]	12

Max RSS (memory usage)

Results (primary -3.9%, secondary -2.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	6.8%	[6.8%, 6.8%]	1
Improvements ✅ (primary)	-3.9%	[-3.9%, -3.9%]	1
Improvements ✅ (secondary)	-6.9%	[-8.8%, -4.9%]	2
All ❌✅ (primary)	-3.9%	[-3.9%, -3.9%]	1

Cycles

Results (secondary 2.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	2.1%	[2.0%, 2.1%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 491.257s -> 490.757s (-0.10%)
Artifact size: 389.43 MiB -> 389.34 MiB (-0.02%)

LorrensP-2158466 · 2026-07-15T15:24:37Z

serde-1.0.219-threads4 regression is gone, thats good at least.

LorrensP-2158466 · 2026-07-15T15:31:24Z

I think that the main bottlenecks are the following (that i notice):

CmRefCell being a RwLock just because of the external resolution tables
get_module taking a lock on the entire table while trying to insert a new external module
the 2 loops in write_import_resolutions

@petrochenkov

…chenkov Resolver: Wrap arenas in `WorkerLocal` In preperation of parallel import resolution rust-lang/rust#158845 this pr wraps the resolver arenas in `WorkerLocal` to ensure we don't need any synchronization to access an arena. This is common in the compiler. Also, when building the resolution table of an external module, we first create a seperate table and fill that one, after which we replace the empty table with that new table. (Prep Work for rust-lang/rust#158845) And a manual inline of `define_extern`. r? @petrochenkov

petrochenkov · 2026-07-16T06:23:21Z

For reference, an attempt to make sync::RwLock "conditionally synchronized" was done in #136772, but it wasn't successful.

@petrochenkov

…tions, r=petrochenkov Resolver: extract `use_injections` out of main `Resolver` Prep work for rust-lang#158845. Extract the `use_injections` field out of the resolver and into the `LateResolutionVisitor`. It can then be passed into the `report_errors` pass, so no behaviour is changed. This allows the `Resolver` to be `DynSync`, see rust-lang#158845 (comment). r? @petrochenkov

LorrensP-2158466 · 2026-07-16T07:42:24Z

For reference, an attempt to make sync::RwLock "conditionally synchronized" was done in #136772, but it wasn't successful.

Ooh, I could try to finish that work.

@petrochenkov

…tions, r=petrochenkov Resolver: extract `use_injections` out of main `Resolver` Prep work for rust-lang#158845. Extract the `use_injections` field out of the resolver and into the `LateResolutionVisitor`. It can then be passed into the `report_errors` pass, so no behaviour is changed. This allows the `Resolver` to be `DynSync`, see rust-lang#158845 (comment). r? @petrochenkov

@petrochenkov

Rollup merge of #159339 - LorrensP-2158466:extract-use-injections, r=petrochenkov Resolver: extract `use_injections` out of main `Resolver` Prep work for #158845. Extract the `use_injections` field out of the resolver and into the `LateResolutionVisitor`. It can then be passed into the `report_errors` pass, so no behaviour is changed. This allows the `Resolver` to be `DynSync`, see #158845 (comment). r? @petrochenkov

[WIP] Benchmarking for the parallel import resolution PR #158845

petrochenkov · 2026-07-16T13:51:10Z

I think that the main bottlenecks are the following (that i notice):

CmRefCell being a RwLock just because of the external resolution tables

get_module taking a lock on the entire table while trying to insert a new external module

the 2 loops in write_import_resolutions

I'll do some additional benchmarking in #159387.

petrochenkov · 2026-07-16T17:25:07Z

The results in #159387 are good, basically we can land that whole PR without making the final change enabling RwLocks. Could you resubmit it (it's your work cherry-picked)?

LorrensP-2158466 · 2026-07-16T18:17:25Z

Will do!

But I must say that I find it weird that it works in multi threaded compilation. The reason for changing it to RwLock was because of the non-synchronized borrow counting of RefCell.

Why does it work now? (Note that I am on my phone, so I could have missed something, but I am curious)

petrochenkov · 2026-07-16T18:30:29Z

Why does it work now?

Because the par_slice call is also removed :D
That's the only removed change besides the rwlocks.

LorrensP-2158466 · 2026-07-17T11:14:39Z

Because the par_slice call is also removed :D

I see it now 😅.

I assume that we will keep this one blocked until we have something for the CmRefCell and the other bottlenecks?

LorrensP-2158466 · 2026-07-17T12:43:43Z

@rustbot blocked on #159440

LorrensP-2158466 · 2026-07-17T15:19:29Z

For reference, an attempt to make sync::RwLock "conditionally synchronized" was done in #136772, but it wasn't successful.

Decided to give it a try in #159452.

rustbot added the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Jul 6, 2026

rustbot assigned petrochenkov Jul 6, 2026

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jul 6, 2026