Analyzer coverage gaps in the Condition Attribute ecosystem

[github-actions]

🎯 Repository Quality Improvement Report — Condition Attribute Ecosystem Analyzer Completeness

Analysis Date: 2026-07-27
Focus Area: Condition Attribute Ecosystem Analyzer Completeness (Custom)
Strategy Type: Custom

Executive Summary

MSTest ships a rich set of declarative skip-condition attributes (OSConditionAttribute, ArchitectureConditionAttribute, CIConditionAttribute, ExecutableConditionAttribute, MemberConditionAttribute) under the shared ConditionBaseAttribute abstraction. The existing analyzers cover several aspects of this system — MSTEST0007 detects condition attributes used without [TestMethod]/[TestClass], MSTEST0041 detects condition attributes on plain (non-[TestClass]) types, MSTEST0061 migrates RuntimeInformation.IsOSPlatform / OperatingSystem.Is* guards to [OSCondition], and MSTEST0070 validates [MemberCondition] member names.

However, a code fix is missing for MSTEST0041, and there are no companion analyzers for the architecture, executable, and CI condition attributes that are parallel to the existing UseOSConditionAttributeInsteadOfRuntimeCheckAnalyzer (MSTEST0061). Additionally, ConditionBaseAttribute silently allows AllowMultiple = true via inheritance while the base AttributeUsage declaration does not explicitly set it, which may confuse users who try to stack multiple conditions.

Full Analysis Report

Focus Area: Condition Attribute Ecosystem Analyzer Completeness

Current State Assessment

Metrics Collected:

Metric	Value	Status
Condition attributes shipped	5 (OSCondition, ArchitectureCondition, CICondition, ExecutableCondition, MemberCondition)	✅
Analyzers covering condition attributes	4 (MSTEST0007, MSTEST0041, MSTEST0061, MSTEST0070)	⚠️
Code fixes for condition-attribute diagnostics	3 (MSTEST0061 ✅, MSTEST0070 ✅ implicit, MSTEST0066 ✅ for [Ignore])	⚠️
Code fix for MSTEST0041	❌ Missing	❌
"UseXCondition instead of imperative check" analyzer for ArchitectureCondition	❌ Missing	❌
"UseXCondition instead of imperative check" analyzer for CICondition	❌ Missing	❌
[Ignore] usages in test/src files	61	⚠️
RuntimeInformation.ProcessArchitecture checks without [ArchitectureCondition]	Multiple (in tests)	⚠️

Findings

Strengths

• OSConditionAttribute has a complete analyzer + code fix (MSTEST0061).
• MemberConditionAttribute has robust compile-time validation of member names (MSTEST0070).
• UseConditionBaseWithTestClassAnalyzer (MSTEST0041) correctly detects condition attributes on non-[TestClass] types.
• UseAttributeOnTestMethodAnalyzer (MSTEST0007) correctly detects condition attributes on methods without [TestMethod].

Areas for Improvement

1. [High] MSTEST0041 has no code fix. The fix is mechanical: add [TestClass] to the decorated type. Without a code fix, developers must manually resolve this common mistake.

2. [Medium] No UseArchitectureConditionAttributeInsteadOfRuntimeCheckAnalyzer. Code like:

   [TestMethod]
   public void MyTest()
   {
       if (RuntimeInformation.ProcessArchitecture != Architecture.X64)
           Assert.Inconclusive("Only for x64");
       // ...
   }

   can be mechanically rewritten to [ArchitectureCondition(TestArchitectures.X64)], exactly analogous to what MSTEST0061 does for OS checks.

3. [Medium] No UseCIConditionAttributeInsteadOfEnvironmentCheckAnalyzer. Patterns like if (Environment.GetEnvironmentVariable("CI") != null) return; inside test methods can be replaced with [CICondition].

4. [Low] The ConditionBaseAttribute AttributeUsage declaration (Inherited = false, no explicit AllowMultiple) makes it unclear whether stacking multiple conditions of the same type on one method is supported. The GroupName-based OR/AND logic does allow multiple attributes, but AllowMultiple = false (the default) would cause a compiler error if a user tries. Since the base sets no AllowMultiple, derived sealed classes that do need stacking (e.g., two [OSCondition] with different modes) silently do not work without users adding AllowMultiple = true to their own custom subclasses.

---

🤖 Suggested Improvement Tasks

The following actionable tasks address the findings above.

Task 1: Add a code fix for MSTEST0041 (UseConditionBaseWithTestClass)

Priority: High
Estimated Effort: Small

MSTEST0041 fires when a ConditionBaseAttribute-derived attribute is applied to a type that is not decorated with [TestClass]. The mechanical fix is to add [TestClass] to the offending type.

Work items:

• Create src/Analyzers/MSTest.Analyzers.CodeFixes/UseConditionBaseWithTestClassFixer.cs.
• The fixer should add [TestClass] to the attribute list of the flagged type declaration.
• Register FixableDiagnosticIds = [DiagnosticIds.UseConditionBaseWithTestClassRuleId].
• Add unit tests in test/UnitTests/MSTest.Analyzers.UnitTests/UseConditionBaseWithTestClassAnalyzerTests.cs for the fix.

---

Task 2: Add UseArchitectureConditionAttributeInsteadOfRuntimeCheckAnalyzer (new rule MSTEST0078)

Priority: Medium
Estimated Effort: Medium

Add an analyzer that detects the following imperative architecture-guard pattern inside [TestMethod] bodies and suggests replacing it with [ArchitectureCondition]:

// Detected patterns:
if (RuntimeInformation.ProcessArchitecture != Architecture.X64)
    Assert.Inconclusive("...");

if (RuntimeInformation.ProcessArchitecture == Architecture.X86)
    return;

Should be replaced with:

[ArchitectureCondition(TestArchitectures.X64)]
[TestMethod]
public void MyTest() { ... }

The implementation can closely mirror UseOSConditionAttributeInsteadOfRuntimeCheckAnalyzer (MSTEST0061) and its companion fixer. Key differences:

• Watch RuntimeInformation.ProcessArchitecture property access rather than IsOSPlatform() / OperatingSystem.Is*() calls.
• Map Architecture enum values to TestArchitectures flags.
• The rule should only apply when ArchitectureConditionAttribute is available (i.e. #if NET guard applies — restrict to non-netfx compilations or guard by type presence).
• Add DiagnosticIds.UseArchitectureConditionAttributeInsteadOfRuntimeCheckRuleId = "MSTEST0078" in DiagnosticIds.cs.
• Add unit tests covering all Architecture → TestArchitectures mappings and both !=-early-return and ==-early-return patterns.

---

Task 3: Add UseCIConditionAttributeInsteadOfEnvironmentCheckAnalyzer (new rule MSTEST0079)

Priority: Medium
Estimated Effort: Medium

Add an analyzer that detects environment-variable checks for common CI indicators inside [TestMethod] bodies:

// Detected patterns:
if (Environment.GetEnvironmentVariable("CI") == null) return;
if (Environment.GetEnvironmentVariable("CI") is null) return;
if (Environment.GetEnvironmentVariable("TF_BUILD") is not null) Assert.Inconclusive("...");

Suggested replacement:

[CICondition]
[TestMethod]
public void MyTest() { ... }

Implementation notes:

• Watch IInvocationOperation on Environment.GetEnvironmentVariable with string arguments matching known CI variables ("CI", "TF_BUILD", "GITHUB_ACTIONS", "BUILD_BUILDID").
• Provide a code fix that removes the guard and adds [CICondition(...)] — with mode derived from the == null (Exclude) vs != null (Include) polarity.
• Add DiagnosticIds.UseCIConditionAttributeInsteadOfEnvironmentCheckRuleId = "MSTEST0079" in DiagnosticIds.cs.
• Add unit tests for both ConditionMode.Include and ConditionMode.Exclude patterns and all recognized CI variable names.

---

Task 4: Document ConditionBaseAttribute.AllowMultiple behavior and add analyzer guard

Priority: Low
Estimated Effort: Small

The ConditionBaseAttribute base class does not set AllowMultiple = true in its [AttributeUsage]. Derived sealed classes (e.g., OSConditionAttribute) also omit it, defaulting to false. This means [OSCondition(OperatingSystems.Windows)][OSCondition(OperatingSystems.Linux)] on the same method will produce a compiler error rather than an OR-combined inclusion list — even though the GroupName API is designed to support exactly this multi-attribute pattern.

Work items:

• Determine the intended design: should stacking the same condition attribute type be supported? If yes, set AllowMultiple = true on the relevant derived attributes (OSConditionAttribute, ArchitectureConditionAttribute) and add acceptance tests that verify OR-group semantics with two attributes of the same type.
• If stacking is intentional, add a PublicAPI.Unshipped.txt entry for the changed AttributeUsage (it changes observable attribute constructor behavior).
• If stacking is deliberately unsupported, add an analyzer or update ConditionBaseAttribute XML docs to clearly state this and recommend [OSCondition(OperatingSystems.Windows | OperatingSystems.Linux)] flags syntax instead.

---

📊 Historical Context

Previous Focus Areas

Date	Focus Area	Type
2026-07-23	parameterized-test-display-name-ux	Custom
2026-07-21	analyzer-code-fix-actionability-gap	Custom
2026-07-20	sourcegen-reflectionfree-coverage-gaps	Custom
2026-07-16	mstest-sdk-msbuild-property-reference-gap	Custom
2026-07-14	assertion-telemetry-coverage-gap	Custom

---

🎯 Recommendations

Immediate Actions (This Week)

1. Add code fix for MSTEST0041 — Priority: High (small effort, high value for IDE users)

Short-term Actions (This Month)

1. Implement MSTEST0078 (UseArchitectureConditionAttributeInsteadOfRuntimeCheck) — Priority: Medium
2. Implement MSTEST0079 (UseCIConditionAttributeInsteadOfEnvironmentCheck) — Priority: Medium
3. Resolve AllowMultiple ambiguity for condition attribute types — Priority: Low

Next analysis: 2026-07-28 — Focus area selected based on diversity algorithm

🤖 Automated content by GitHub Copilot. Generated by the Repository Quality Improver workflow. · sonnet46 78.6 AIC · ⌖ 5.17 AIC · ⊞ 10.7K · [◷]( · ◷)

_{Add this agentic workflow to your repo}

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repository-quality-improver.md@main

• expires on Jul 29, 2026, 10:40 PM UTC