fix: schema backwards compat for attributes_skipped key

[jeremyeder]

Summary

• Assessment.to_dict() now serializes attributes_skipped to match the JSON schema (was outputting attributes_not_assessed)
• Schema updated with anyOf to accept either attributes_skipped or attributes_not_assessed for backwards compatibility with existing reports
• Learning service fallback order updated to check canonical key first

Context

Leaderboard validation was failing because submitted reports used attributes_not_assessed but the schema required attributes_skipped. This fixes both directions: new reports use the canonical key, and old reports still validate.

Test Plan

• Schema validator tests pass with both key names
• Learning service tests pass (including backwards compat test)
• Model tests pass
• Manual verification: old key validates, new key validates, neither key fails

🤖 Generated with Claude Code

[github-actions]

📈 Test Coverage Report

Branch	Coverage
This PR	65.2%
Main	65.2%
Diff	✅ +0%

---

Coverage calculated from unit tests only

[github-actions]

AgentReady Code Review - PR #277

Executive Summary

Overall Assessment: ✅ APPROVED - Excellent backwards compatibility fix with strong test coverage

Impact on AgentReady Score: Neutral to positive (fixes schema validation, improves interoperability)

---

🎯 Attribute Compliance Analysis

✅ Strong Performance

Attribute	Status	Evidence
Type Annotations	✅ PASS	All modified code maintains proper Python type hints (dict, list[str], int)
Test Coverage	✅ PASS	New backwards compatibility test added (test_extract_patterns_with_old_schema_key)
Code Quality	✅ PASS	Clean implementation, follows existing patterns, minimal changes
Documentation	✅ PASS	Clear PR description, inline comments updated to match changes
Schema Versioning	✅ PASS	Proper use of JSON Schema anyOf for backwards compatibility

📋 Technical Review

Schema Design (assessment-schema.json:10-17, 60-71)

✅ Excellent backwards compatibility pattern

Changes:

• Moved attributes_skipped from required array to anyOf constraint
• Added attributes_not_assessed as deprecated alias
• Clear descriptions distinguish canonical vs deprecated keys

Strengths:

• JSON Schema best practice: anyOf allows either key to satisfy validation
• Both old and new reports will validate successfully
• Clear documentation of canonical key vs deprecated alias

Recommendation: Consider adding deprecation notice in schema description for future removal planning (e.g., "will be removed in v2.0.0")

Model Serialization (assessment.py:91)

✅ Correct key mapping

- "attributes_not_assessed": self.attributes_not_assessed,
+ "attributes_skipped": self.attributes_not_assessed,

Strengths:

• Internal model property name (attributes_not_assessed) remains unchanged (good for API stability)
• Only serialization output changes (good separation of concerns)
• No breaking changes to Python API

Note: The internal property name could be renamed in a future major version for consistency, but leaving it unchanged is the right call for backwards compatibility.

Deserialization Fallback (learning_service.py:148-150)

✅ Robust fallback handling

attributes_not_assessed=assessment_data.get(
    "attributes_skipped", assessment_data.get("attributes_not_assessed", 0)
)

Strengths:

• Checks canonical key first (attributes_skipped)
• Falls back to old key (attributes_not_assessed)
• Defaults to 0 if neither exists (graceful degradation)
• Nested .get() pattern is clear and idiomatic

Alternative consideration: Could use a helper function for readability:

def _get_attributes_skipped(data: dict) -> int:
    """Get skipped attributes count, checking both key names."""
    return data.get("attributes_skipped", data.get("attributes_not_assessed", 0))

Not required for this PR - just a suggestion for future refactoring

Test Coverage (test_learning_service.py)

✅ Comprehensive backwards compatibility testing

Added test: test_extract_patterns_with_old_schema_key

• Validates old key name (attributes_not_assessed) still works
• Comment updated to clarify which key is old vs new (line 355)

Strengths:

• Tests the exact backwards compatibility scenario
• Clear test naming and documentation
• Fixtures updated to use canonical key

---

🔒 Security Analysis

✅ No security issues identified

• No injection vulnerabilities
• No exposure of sensitive data
• Schema validation still enforces proper types and ranges
• No changes to authentication/authorization logic

---

🏆 Best Practices Assessment

✅ Excellent Execution

1. Minimal Changeset: Only 5 files modified, focused changes
2. Test-First Approach: Tests updated alongside implementation
3. Clear Documentation: PR body explains problem, solution, and test plan
4. Backwards Compatibility: Proper migration path for existing data
5. Schema Evolution: Follows semantic versioning principles

---

📊 Impact on AgentReady Self-Assessment

Projected Score Impact: +0.5 points (80.0 → 80.5)

Improvements:

• Schema Versioning attribute strengthened (proper backwards compatibility handling)
• API Stability improved (old reports still work)
• Test Coverage maintained (backwards compat test added)

Affected Attributes:

• schema_versioning: Score increases from partial to full compliance
• testing_framework: Already strong, maintains coverage
• documentation_clarity: PR description is exemplary

---

🎬 Recommendations

Required Before Merge: None ✅

All checks pass, implementation is production-ready.

Future Enhancements (Optional):

1. Deprecation Timeline: Add schema comment indicating when attributes_not_assessed will be removed (v2.0.0?)
2. Metrics Tracking: Consider logging which key is being used to track migration progress
3. Property Naming Consistency: In a future major version, consider renaming the internal attributes_not_assessed property to attributes_skipped for full consistency

Example Deprecation Warning (for future PR):

"attributes_not_assessed": {
  "type": "integer",
  "minimum": 0,
  "maximum": 25,
  "description": "DEPRECATED: Use 'attributes_skipped' instead. This key will be removed in v2.0.0.",
  "deprecated": true
}

---

✅ Final Verdict

APPROVED - Merge when CI passes

Rationale:

• Solves real leaderboard validation issue
• Zero breaking changes
• Excellent test coverage
• Follows JSON Schema best practices
• Clear migration path for legacy data
• Maintains AgentReady's high code quality standards

Blocking Issues: None
Non-Blocking Suggestions: 3 (all future enhancements)

---

📚 References

• JSON Schema anyOf Pattern
• Semantic Versioning
• AgentReady Schema Contract

---

Review performed by AgentReady Code Review Agent
Timestamp: 2026-02-06
Review Scope: Full PR diff, schema validation, backwards compatibility

[github-actions]

🎉 This PR is included in version 2.25.2 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀