You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Recent PRs #9514 and #9617 eliminated significant allocations in data-driven test execution:
#9514: cached MethodInfo.GetParameters() in TestMethodInfo.ParameterTypes
#9617: eliminated O(N) allocations across data row iteration — CloneForDataDrivenIteration dictionary, TCS bridge fast path, ReflectionTestMethodInfo wrapper caching
The existing performance runner (test/Performance/MSTest.Performance.Runner/) only has Scenario1: 100 classes × 100 plain test methods (10,000 total). There is no scenario that exercises data-driven tests, so there is no regression baseline for the improvements in #9514 and #9617.
Proposed: Scenario2
Add a new scenario that measures data-driven test execution performance:
Suggested shape:
10 classes × 10 test methods × 100 data rows each = 10,000 test invocations
Each method attributed with [DataRow] (100 rows) or a [DynamicData] source
Mirrors total invocation count of Scenario1 so results are directly comparable
What this enables:
Baseline measurement for the allocation improvements already merged
Regression protection for future changes to data-driven hot paths (TestMethodRunner, ReflectionTestMethodInfo, TestMethodInfo.ResolveArguments)
Comparison of wall-clock time and CPU time between plain and data-driven execution models (helpful for diagnosing overhead sources)
Why now
Without this scenario, any regression in data-driven allocation paths would go undetected. The infrastructure already supports this — Scenario1 demonstrates the pattern, and the runner is designed to be extended.
Acceptance criteria
Scenario2.cs added under test/Performance/MSTest.Performance.Runner/Scenarios/
Scenario runs end-to-end with PlainProcess and/or DotnetTestProcess
Results are comparable in format to Scenario1 output (JSON + compressed ZIP)
Background
Recent PRs #9514 and #9617 eliminated significant allocations in data-driven test execution:
#9514: cachedMethodInfo.GetParameters()inTestMethodInfo.ParameterTypes#9617: eliminated O(N) allocations across data row iteration —CloneForDataDrivenIterationdictionary, TCS bridge fast path,ReflectionTestMethodInfowrapper cachingThe existing performance runner (
test/Performance/MSTest.Performance.Runner/) only hasScenario1: 100 classes × 100 plain test methods (10,000 total). There is no scenario that exercises data-driven tests, so there is no regression baseline for the improvements in #9514 and #9617.Proposed: Scenario2
Add a new scenario that measures data-driven test execution performance:
Suggested shape:
[DataRow](100 rows) or a[DynamicData]sourceWhat this enables:
TestMethodRunner,ReflectionTestMethodInfo,TestMethodInfo.ResolveArguments)Why now
Without this scenario, any regression in data-driven allocation paths would go undetected. The infrastructure already supports this —
Scenario1demonstrates the pattern, and the runner is designed to be extended.Acceptance criteria
Scenario2.csadded undertest/Performance/MSTest.Performance.Runner/Scenarios/PlainProcessand/orDotnetTestProcessRelated
test/Performance/MSTest.Performance.Runner/Scenarios/Scenario1.csAdd this agentic workflows to your repo
To install this agentic workflow, run