Skip to content

[efficiency-improver] Add Scenario2: data-driven test performance benchmark #9713

Description

@github-actions

Background

Recent PRs #9514 and #9617 eliminated significant allocations in data-driven test execution:

  • #9514: cached MethodInfo.GetParameters() in TestMethodInfo.ParameterTypes
  • #9617: eliminated O(N) allocations across data row iteration — CloneForDataDrivenIteration dictionary, TCS bridge fast path, ReflectionTestMethodInfo wrapper caching

The existing performance runner (test/Performance/MSTest.Performance.Runner/) only has Scenario1: 100 classes × 100 plain test methods (10,000 total). There is no scenario that exercises data-driven tests, so there is no regression baseline for the improvements in #9514 and #9617.

Proposed: Scenario2

Add a new scenario that measures data-driven test execution performance:

Suggested shape:

  • 10 classes × 10 test methods × 100 data rows each = 10,000 test invocations
  • Each method attributed with [DataRow] (100 rows) or a [DynamicData] source
  • Mirrors total invocation count of Scenario1 so results are directly comparable

What this enables:

  • Baseline measurement for the allocation improvements already merged
  • Regression protection for future changes to data-driven hot paths (TestMethodRunner, ReflectionTestMethodInfo, TestMethodInfo.ResolveArguments)
  • Comparison of wall-clock time and CPU time between plain and data-driven execution models (helpful for diagnosing overhead sources)

Why now

Without this scenario, any regression in data-driven allocation paths would go undetected. The infrastructure already supports this — Scenario1 demonstrates the pattern, and the runner is designed to be extended.

Acceptance criteria

  1. Scenario2.cs added under test/Performance/MSTest.Performance.Runner/Scenarios/
  2. Scenario runs end-to-end with PlainProcess and/or DotnetTestProcess
  3. Results are comparable in format to Scenario1 output (JSON + compressed ZIP)
  4. README or comment explains the scenario shape and its relationship to [efficiency-improver] perf: cache MethodInfo.GetParameters() to avoid per-row array allocations in data-driven tests #9514/perf: reduce allocations in data-driven test execution hot path #9617

Related

🤖 Automated content by GitHub Copilot. Generated by the Efficiency Improver workflow. · 351.3 AIC · ⌖ 19.3 AIC · ⊞ 13.6K · [◷]( · )

Add this agentic workflows to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/efficiency-improver.md@main

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/performanceRuntime / build performance / efficiency.type/automationCreated or maintained by an agentic workflow.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions