Skip to content

Policy Evaluation Engine #785

Description

@malibio

Overview

Implement a Policy Evaluation Engine that periodically scans nodes and executes lifecycle/retention policy transitions. This is the runtime component that makes policies actionable.

Architecture Context

This issue is part of the Universal Graph Architecture Transition (Epic #783).

Policies are declarative rules stored as nodes. The Policy Evaluation Engine is the runtime that:

  1. Reads enabled policies
  2. Finds nodes matching policy triggers
  3. Executes the specified actions (archive, delete, purge)
  4. Optionally triggers workflows for side effects

Problem Statement

With policy nodes defined (#755), we need a system to actually evaluate and execute them. Without this:

  • Lifecycle policies exist but do nothing
  • Users must manually archive/delete content
  • Retention policies can't auto-purge old deleted content

Proposed Solution

1. Engine Architecture

┌─────────────────────────────────────────────────────────┐
│                 Policy Evaluation Engine                │
├─────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐     │
│  │   Loader    │  │  Evaluator  │  │  Executor   │     │
│  │             │  │             │  │             │     │
│  │ Read active │  │ Find nodes  │  │ Transition  │     │
│  │ policies    │  │ matching    │  │ lifecycle   │     │
│  │             │  │ triggers    │  │ states      │     │
│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘     │
│         │                │                │             │
│         ▼                ▼                ▼             │
│  ┌──────────────────────────────────────────────┐      │
│  │              Scheduler (Background Job)       │      │
│  │  - Runs every N minutes (configurable)        │      │
│  │  - Respects app idle state                    │      │
│  │  - Batches operations to avoid blocking       │      │
│  └──────────────────────────────────────────────┘      │
└─────────────────────────────────────────────────────────┘

2. Policy Types Supported

policy_type Trigger Types Actions
lifecycle last_modified, created_at, last_accessed archive, delete
retention deleted_at, archived_at purge (hard delete)

3. Trigger Conditions

{
  "trigger": {
    "type": "last_modified",
    "days": 90,
    "condition": "properties.status == 'done'"
  }
}
  • type: Which timestamp to check
  • days: How old before triggering
  • condition: Optional SurrealQL/expression filter

4. Evaluation Query Example

-- Find tasks to archive (done + not modified in 90 days)
SELECT id FROM node 
WHERE node_type IN $applies_to
  AND lifecycleStatus = 'active'
  AND properties.status == 'done'
  AND modified_at < time::now() - 90d
LIMIT 100;  -- Batch size

5. Execution Flow

  1. Load enabled policies ordered by priority
  2. Evaluate each policy, collecting matching node IDs
  3. Execute state transitions in batches
  4. Emit events for workflow triggers (if configured)
  5. Log actions for audit trail

6. Safety Constraints

  • Batch size: Max 100 nodes per execution cycle
  • Idle requirement: Only run when app idle > 30 seconds
  • Rate limit: Max 1 evaluation cycle per minute
  • Dry run mode: Preview what would change without executing

Acceptance Criteria

Phase 1: Core Engine

  • Policy Loader reads enabled policies from database
  • Policy Evaluator builds and executes match queries
  • Policy Executor transitions node lifecycle states
  • Engine runs as background job on configurable interval

Phase 2: Lifecycle Policies

  • last_modified trigger type works
  • created_at trigger type works
  • condition expressions evaluated correctly
  • archive action transitions to lifecycleStatus: 'archived'
  • delete action transitions to lifecycleStatus: 'deleted'

Phase 3: Retention Policies

  • deleted_at trigger type works (time since deleted)
  • purge action hard-deletes nodes
  • Purge respects any child relationships (cascade or block)

Phase 4: Safety & Observability

  • Batch size limits enforced
  • Idle detection prevents running during active use
  • Dry run mode for policy testing
  • Audit log of all policy executions
  • Metrics: policies evaluated, nodes transitioned, errors

Phase 5: Workflow Integration

  • Policy execution can emit events
  • Workflows can subscribe to lifecycle transition events
  • Example: "When archived, notify owner" workflow works

Phase 6: Quality

  • Documentation updated
  • Tests cover all trigger types and actions
  • Tests cover batch limits and error handling
  • Code passes bun run quality:fix
  • Tests pass (bun run test:all)

Technical Specifications

Reference Files

Scheduling Options

Trigger Behavior
Interval Every 5 minutes (default)
App idle After 30 seconds of no user activity
Manual User triggers "Run policies now"
Startup Light evaluation on app launch

Condition Expression Syntax

Use a subset of SurrealQL or a simple expression language:

properties.status == 'done'
properties.priority IN ['low', 'medium']
content CONTAINS 'draft'

Dependencies

Related Issues

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions