Skip to content

feat: quality profiles — preset and configurable rule sets per project #208

@ajianaz

Description

@ajianaz

Summary

Add Quality Profiles — preset and configurable rule sets that define what Cora focuses on during review. Different projects need different review strictness: a security-critical API needs stricter review than a prototype experiment.

Motivation

  • SonarQube Quality Profiles are the .cora.yaml config system (per-project, hierarchical) #3 most-used feature — teams manage per-language, per-project rule sets
  • Currently Cora has no preset system — every project manually configures .cora.yaml from scratch
  • Discoverability problem — new users don't know what to put in .cora.yaml. Presets give them a starting point
  • Consistency across repos — team can share one profile and apply it to all repos

Proposed Design

Built-in Profiles

$ cora profile list

Available profiles:
  ┌─────────────────┬─────────────────────────────────────────────────┐
  │ Profile          │ Description                                    │
  ├─────────────────┼─────────────────────────────────────────────────┤
  │ security-first  │ Strict security focus — zero tolerance for      │
  │                  │ vulnerabilities, secrets, injection risks       │
  ├─────────────────┼─────────────────────────────────────────────────┤
  │ performance     │ Focus on speed, memory, allocation patterns     │
  │                  │ Best for hot-path and critical section code     │
  ├─────────────────┼─────────────────────────────────────────────────┤
  │ clean-code      │ Broad quality — readability, naming, complexity │
  │                  │ Best for team projects with style standards     │
  ├─────────────────┼─────────────────────────────────────────────────┤
  │ beginner-friendly│ Gentle review — focus on common mistakes,      │
  │                  │ learning opportunities, explanatory comments    │
  ├─────────────────┼─────────────────────────────────────────────────┤
  │ minimal         │ Only critical + security, nothing else          │
  │                  │ Best for quick PRs and hotfixes                 │
  ├─────────────────┼─────────────────────────────────────────────────┤
  │ rust-strict     │ Rust-specific: unsafe, unwrap, panic, lifetime  │
  │                  │ error handling, idiomatic patterns              │
  ├─────────────────┼─────────────────────────────────────────────────┤
  │ typescript-strict│ TS-specific: any types, null safety, proper     │
  │                  │ typing, async patterns                          │
  ├─────────────────┼─────────────────────────────────────────────────┤
  │ go-pragmatic    │ Go-specific: error handling, goroutine safety   │
  │                  │ interface design, idiomatic Go                  │
  └─────────────────┴─────────────────────────────────────────────────┘

Profile Definition Format

Each profile is a YAML file that defines focus areas, weights, and review behavior:

# profiles/security-first.yaml
name: security-first
description: "Strict security focus — zero tolerance for vulnerabilities"
version: "1.0"

focus_areas:
  - id: security
    weight: 10        # review priority (1-10)
    action: block     # block = any finding → fail
    rules:
      - "No hardcoded secrets or credentials"
      - "Validate all external inputs"
      - "Use parameterized queries, never string interpolation"
      - "Check authorization on every endpoint"
      - "Sanitize user-generated content"

  - id: injection
    weight: 10
    action: block
    rules:
      - "No SQL/NoSQL/XSS/SSRF injection vectors"
      - "Escape all output in templates"
      - "Use ORM/query builder, never raw queries"

  - id: error_handling
    weight: 7
    action: warn
    rules:
      - "Never expose stack traces to users"
      - "Log security events properly"
      - "Handle all error paths explicitly"

ignore_areas:
  - style
  - naming
  - documentation

severity_override:
  # Upgrade severity for specific patterns
  hardcoded_value: critical    # normally medium → critical in security profile
  todo_comment: ignore         # normally low → ignore

review_style:
  tone: strict                 # strict | standard | gentle
  detail_level: high           # minimal | standard | high | exhaustive
  suggest_fixes: true          # always suggest remediation
  max_findings: null           # no limit — report everything

.cora.yaml Usage

# Option 1: Use built-in profile
profile: security-first

# Option 2: Extend built-in profile with overrides
profile:
  extends: security-first
  overrides:
    focus_areas:
      - id: compliance
        weight: 8
        action: warn
        rules:
          - "Include data processing consent checks"
          - "Log PII access events"
    ignore_areas:
      - style

# Option 3: Custom profile from file
profile: ./my-team-profile.yaml

# Option 4: Inline custom (no file needed)
profile:
  name: "trapfall-security"
  focus_areas:
    - id: unsafe_rust
      weight: 10
      action: block
      rules:
        - "No unsafe blocks without safety comment"
        - "No unwrap() in production code paths"
    - id: crypto
      weight: 10
      action: block
      rules:
        - "Use constant-time comparison for secrets"
        - "Never roll custom crypto"
  ignore_areas:
    - documentation
    - test_style

CLI Commands

# List available profiles
cora profile list

# Show profile details
cora profile show security-first

# Validate a custom profile
cora profile validate ./my-profile.yaml

# Generate a profile interactively
cora profile init
# → What language? [Rust/TypeScript/Go/Python/...]
# → Primary focus? [Security/Performance/Clean Code/All]
# → Strictness? [Strict/Standard/Gentle]
# → Generated: .cora/profile.yaml

# Preview what a profile would flag (dry-run against existing codebase)
cora profile preview security-first

Profile Resolution Order

1. Inline .cora.yaml profile definition     (highest priority)
2. Custom profile file path in .cora.yaml
3. Profile extends + overrides              (merge with base)
4. Built-in profile name in .cora.yaml
5. Auto-detect from language + project      (fallback)

Auto-Detection

If no profile is specified, Cora auto-detects based on project language:

Detected Language Default Profile
Rust rust-strict
TypeScript/JavaScript typescript-strict
Go go-pragmatic
Python clean-code
Unknown/Mixed clean-code

Profile Effect on AI Prompt

The profile directly modifies the review prompt sent to the LLM:

Without profile:  "Review this diff for bugs, security issues, and code quality."

With profile:     "Review this diff using the 'security-first' profile:
                   - CRITICAL focus: security, injection prevention (weight: 10)
                   - HIGH focus: error handling (weight: 7)
                   - IGNORE: style, naming, documentation
                   - Tone: strict, detail_level: high
                   - Always suggest fixes
                   - Block on any security finding"

This makes the AI review targeted and consistent rather than generic.

Architecture

src/
├── engine/
│   ├── profiles.rs           # NEW: Profile loading, validation, resolution
│   │   ├── struct Profile
│   │   ├── struct FocusArea
│   │   ├── fn load_profile(config) -> Profile
│   │   ├── fn resolve_profile(inline, extends, builtin) -> Profile
│   │   ├── fn validate_profile(yaml) -> Result
│   │   └── fn auto_detect(language) -> Profile
│   └── ...
├── commands/
│   └── profile.rs            # NEW: `cora profile` subcommand group
├── profiles/                 # NEW: Built-in profile YAML files
│   ├── security-first.yaml
│   ├── performance.yaml
│   ├── clean-code.yaml
│   ├── beginner-friendly.yaml
│   ├── minimal.yaml
│   ├── rust-strict.yaml
│   ├── typescript-strict.yaml
│   └── go-pragmatic.yaml
└── engine/
    └── prompt_builder.rs     # MODIFY: Inject profile rules into AI prompt

Implementation Checklist

  • Define Profile and FocusArea structs in src/engine/profiles.rs
  • Create built-in profile YAML files (8 profiles)
  • Implement profile loading from .cora.yaml (inline, path, extends)
  • Implement built-in profile resolution by name
  • Implement profile inheritance (extends + overrides merge)
  • Implement auto-detection based on project language
  • Implement profile validation (schema check)
  • Modify prompt builder to inject profile rules into AI review prompt
  • Create src/commands/profile.rs with list, show, validate, init subcommands
  • Add cora profile init interactive wizard
  • Add cora profile preview dry-run mode
  • Include built-in profiles in binary (embed with include_str!)
  • Write unit tests for profile resolution and merging
  • Write integration tests per built-in profile
  • Update README with profile documentation

Out of Scope (Future)

  • Community profile sharing/registry
  • Profile marketplace
  • Organization-level profiles (shared across repos via GitHub App)
  • Profile analytics (which rules fire most)

Dependencies

  • Quality Gate (Issue #N) — profiles define which categories trigger gate failure
  • MCP Server (Issue #N) — cora.list_rules exposes profile rules to AI agents
  • Rules engine (existing v0.4) — profiles layer on top of deterministic rules

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions