Close SDK-side gaps in MTP artifact post-processing

[Evangelink]

Follow-up to #55419 (the feature) and #55480 (hardening). Artifact post-processing shipped as SDK orchestration plus a TRX post-processor, but a handful of SDK-owned gaps against microsoft/testfx RFC 018 were left open. This closes them.

A truncated run could be merged into one authoritative-looking report

Only Ctrl+C skipped post-processing. A run stopped by --maximum-failed-tests or --timeout still merged whatever artifacts existed: modules that never started contributed nothing, and modules killed mid-flight wrote whatever they had. The merged report then looked like a complete run. Both cases now skip, exactly as Ctrl+C already did.

The decision moved out of the inline condition into ShouldPostProcessArtifacts, so all five reasons to skip (help, discovery, opt-out, cancellation, policy truncation) are covered by deterministic unit tests instead of only by a timing-dependent end-to-end test.

The merge host did not see --config-file

A merge host resolves and enables its extensions exactly like a test host does, so a configuration file that governs which extensions load has to reach it. Without it the merge ran with a different extension set than the run that produced the artifacts, and a post-processor disabled by configuration silently came back for the merge. --config-file is now forwarded alongside --diagnostic-output-directory.

--results-directory is deliberately still not forwarded: the merged output location travels in the manifest so the SDK keeps control of it even when it has to be derived. That is now stated in a comment rather than being implicit.

Merged output landed in an arbitrary project's bin folder

Without --results-directory the output directory was the directory of whichever input sorted first. It is now the directory of an input produced by the elected application when there is one, so the merged artifact lands beside the reports it summarizes rather than inside an unrelated project's output. Path order remains the fallback, because an application can be elected purely for the kinds it advertises without having produced any input.

Other gaps

• Configurable timeout. The 15 minute bound on a merge host is now overridable through DOTNET_CLI_TEST_ARTIFACT_POST_PROCESSING_TIMEOUT_SECONDS, with 0 removing it entirely (which is what makes attaching a debugger to a merge host possible). Values the framework would reject, above roughly 49.7 days, collapse to "no bound" instead of throwing out of the wait and failing every merge.
• Progress feedback. Post-processing runs after the last module exits and before the summary, so a slow merge was indistinguishable from a hung CLI. A single line is printed once a plan exists.
• Telemetry (RFC section 7.11). New test/artifact-post-processing event with jobs planned/executed/failed, artifact count, kinds, extensions and wall time. Kinds and extensions are producer-chosen strings, so anything outside a fixed well-known list is reported as other and a private post-processor's identifier is never uploaded. Runs that plan nothing emit nothing.
• Documentation. The feature had none. Adds documentation/general/dotnet-test-artifact-post-processing.md covering when it runs and when it is skipped, how groups and elections are computed, where merged output lands, the --no-build caveat, why this is unrelated to VSTest's VSTEST_DISABLE_ARTIFACTS_POSTPROCESSING, and troubleshooting. Plus the telemetry event entry.

Hardening found in review

ExecuteAsync guarded each job but not the code around them. Planning, the progress line and the telemetry call sat outside any catch, and ExecuteAsync is invoked with .GetAwaiter().GetResult() under a try/finally with no catch, so a throw from any of them could still turn a finished run into a CLI crash with a different exit code. The whole body is now guarded. Separately, a merge killed by Ctrl+C reported jobs_failed in telemetry even though the user-facing warning was deliberately suppressed for that case; ReportFailureUnlessCancelled now returns whether it reported, and the counter follows it.

Pre-existing test failure fixed

MultiProjectRun_MergesTrxArtifacts was failing on main. I confirmed this by stashing everything and rebuilding before touching it.

This suite runs under Microsoft.Testing.Platform, so its process already carries a TESTINGPLATFORM_DOTNETTEST_EXECUTIONID. A test application only generates its own when that variable is unset, so the applications launched by the inner dotnet test adopted the harness's id and both invocations looked like the same execution. The merged report name is derived from the inputs and their execution ids, so the second run collided with the first. The test now clears the variable per invocation.

Worth flagging as a possible follow-up, deliberately not changed here: this means dotnet test propagates an inherited execution id to every test application it starts, contrary to the platform's documented "each test application gets its own" model. It is niche and upstream-adjacent, so I did not change that hot path.

Still out of scope

Coverage merging needs a post-processor in the extension that produces .coverage (RFC Phase 3, microsoft/codecoverage); coverlet is Phase 4; de-duplicating retried tests belongs to the merge engine. Until Phase 3 ships, TRX remains the only format that consolidates, which is why #47613 stays open.

Validation

build.cmd -c Debug is clean. 52/52 ArtifactPostProcessing* tests pass, plus 89 neighbouring tests (TerminalTestReporter, TestApplicationHandler, TestProgressState, MTPHelpSnapshot, TestCommandParser, and the max-failed-tests, timeout and retry end-to-end tests). All 15 .xlf files were regenerated by the build rather than hand-edited.

New coverage: 2 planner tests (kinds sharing an extension are not conflated; a tagged artifact is never routed through both its kind group and the extension fallback), 11 manager tests (output directory selection, timeout parsing, merge-host argument construction, the skip decision), 3 telemetry tests including one asserting no path or project name reaches any property, and 3 end-to-end tests from the RFC behavior matrix (single application produces no merged report, a run truncated by --maximum-failed-tests is not merged, --test-modules merges and the merged counters equal the sum of the per-module ones).

Relates to #47613.

[azure-pipelines]

Azure Pipelines:
Successfully started running 2 pipeline(s).
1 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Closes remaining SDK-owned gaps in Microsoft.Testing.Platform (MTP) artifact post-processing for dotnet test, aligning behavior with the intended “best-effort, never misrepresent a truncated run” contract and adding missing observability/documentation.

Changes:

• Skip artifact post-processing for truncated runs (--maximum-failed-tests, --timeout, Ctrl+C) via a centralized ShouldPostProcessArtifacts decision with unit coverage.
• Improve merge-host behavior and UX: forward --config-file, choose a more appropriate merged output directory, add a “merge starting” progress line, and make the merge-host timeout configurable via env var.
• Add telemetry (test/artifact-post-processing) + docs describing behavior, troubleshooting, and knobs.

Show a summary per file

File	Description
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsArtifactPostProcessingMTP.cs	Adds E2E coverage for “single app => no merge”, “max-failed-tests => skip merge”, --test-modules merge, and attempts to isolate execution-id behavior.
test/dotnet.Tests/CommandTests/Test/ArtifactPostProcessingTelemetryTests.cs	Adds unit coverage ensuring telemetry shape/duration reporting and bucketing of unknown kinds/extensions (privacy).
test/dotnet.Tests/CommandTests/Test/ArtifactPostProcessingPlannerTests.cs	Adds planner regressions for kind-vs-extension grouping and ensuring tagged artifacts aren’t double-routed.
test/dotnet.Tests/CommandTests/Test/ArtifactPostProcessingManagerTests.cs	Adds tests for merge-host argument forwarding, timeout parsing, skip decision logic, and merged output directory selection.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.zh-Hant.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.zh-Hans.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.tr.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ru.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.pt-BR.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.pl.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ko.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.ja.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.it.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.fr.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.es.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.de.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/xlf/CliCommandStrings.cs.xlf	Adds localization entry for the new “post-processing started” progress line.
src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs	Extracts merge-host argument building (incl. --config-file), and adds configurable merge timeout parsing.
src/Cli/dotnet/Commands/Test/MTP/Terminal/TerminalTestReporter.cs	Adds an information-message writer used to display merge-start progress.
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs	Centralizes skip logic for post-processing into ShouldPostProcessArtifacts.
src/Cli/dotnet/Commands/Test/MTP/ArtifactPostProcessingTelemetry.cs	Implements telemetry event/property creation with bucketing to avoid uploading private identifiers.
src/Cli/dotnet/Commands/Test/MTP/ArtifactPostProcessingManager.cs	Adds progress line, telemetry emission, stronger outer exception guard, better output directory selection, and improves failure counting semantics.
src/Cli/dotnet/Commands/Test/CliConstants.cs	Introduces env var name for configuring the merge-host timeout.
src/Cli/dotnet/Commands/CliCommandStrings.resx	Adds the localized string for the merge-start progress line.
documentation/project-docs/telemetry.md	Documents the new test/artifact-post-processing telemetry event.
documentation/general/dotnet-test-artifact-post-processing.md	Adds end-user documentation for when post-processing runs, skip conditions, configuration, and troubleshooting.

Copilot's findings

• Files reviewed: 26/26 changed files
• Comments generated: 2