[Benchmarks] Make partitions default to number of cores instead of 2#8292
Merged
Dandandan merged 2 commits intoapache:mainfrom Nov 22, 2023
Merged
[Benchmarks] Make partitions default to number of cores instead of 2#8292Dandandan merged 2 commits intoapache:mainfrom
partitions default to number of cores instead of 2#8292Dandandan merged 2 commits intoapache:mainfrom
Conversation
Dandandan
approved these changes
Nov 21, 2023
viirya
approved these changes
Nov 21, 2023
Contributor
|
We are going to need some bigger benchmarks :) -- by the time the queries are in the 250ms range, there is a lot of noise with whatever else is going on in the system :) |
alamb
approved these changes
Nov 21, 2023
| for i in 0..self.common.iterations { | ||
| let config = | ||
| SessionConfig::new().with_target_partitions(self.common.partitions); | ||
| let config = SessionConfig::new().with_target_partitions( |
Contributor
There was a problem hiding this comment.
I think SessionConfig already defaults to the number of CPUs, so we can probably just avoid setting this at all if self.common.partitions.is_none()
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Related to discussion in #7001
Rationale for this change
By default, the benchmarks have been running with two partitions (i.e., concurrency of two), which is unrealistic given the number of cores on modern laptops. This PR defaults to num cores instead. We can still override this and explicitly set the number of partitions.
This is the difference this makes for me (Threadripper with 24 cores).
What changes are included in this PR?
Are these changes tested?
Are there any user-facing changes?
No