Epic: Universal Graph Architecture Transition

[malibio]

Overview

NodeSpace is transitioning from a Hub-and-Spoke architecture to a Universal Graph Architecture with Pure JSON Properties. This epic tracks all work required for this transition.

Strategic Goals

This architecture supports NodeSpace's core differentiators:

1. AI-Native Context: Agents traverse arbitrary relationships via MCP without schema friction
2. Playbook Marketplace: Users install new methodologies (Schema-as-Node) without database migrations
3. Local-First Performance: Zero-latency local reads with SurrealDB Cloud handling sync and RLS
4. Simplified Sync: Single-record atomic operations eliminate "ghost node" problems

Architecture Documents

• Target State: docs/architecture/architecture-target-state.md
• Archived: docs/architecture/archived/architecture-overview-2025-hub-spoke.md
• Archived: docs/architecture/archived/surrealdb-schema-design-hub-spoke.md

Key Changes

Component	Before	After
Node Storage	Hub table + Spoke tables (task, schema)	Single node table with properties JSON
Type-Specific Data	Record Links to spoke tables	properties field with partial indexes
Schema Storage	schema spoke table	Nodes with node_type = 'schema'
Query Pattern	N+1 (fetch node, then fetch spoke)	Single query
Sync Complexity	2 tables atomic	1 record atomic
New Types	Requires CREATE TABLE	Zero DDL

Work Streams

Stream A: Foundation (Prerequisites) ✅

• Remove Hardcoded Node Type Validation - Enable Schema-Based Type Discovery #446 - Remove Hardcoded Node Type Validation (COMPLETED)

Stream B: Data Model Transition (READY TO START)

Eliminate spoke tables and move to Pure JSON:

• Remove data field (Record Link to spoke)
• Migrate spoke data to node.properties
• Delete spoke tables (task, schema spokes)
• Add partial indexes for queryable properties
• Simplify SchemaTableManager to index management only
• Remove N+1 property fetching code (~500 LOC deletion)
• Consolidate Edge Tables into Universal Edge Table #788 - Consolidate Edge Tables into Universal Edge Table

Stream C: Security Layer

Implement ABAC + RLS for cloud sync:

• Add type field protection (prevent permission bypass)
• Implement type_grants on user records
• Define PERMISSIONS clauses on node table
• Protect schema nodes from non-admin modification

Stream D: Sync Infrastructure

Prepare for SurrealDB Cloud sync:

• SurrealDB Cloud PoC (separate repo) - see docs/architecture/experiments/surrealdb-cloud-poc-plan.md
• Implement conflict detection (OCC via version field)
• Implement "Conflicted Copies" resolution strategy
• Build offline operation queue
• Sync Policy Infrastructure #784 - Sync Policy Infrastructure (pull/push/eviction policies)

Stream E: Governance & Lifecycle

Policy-driven content governance:

• Add lifecycle_status as Core Node Property with Semantic Search Exclusion #755 - Add lifecycle_status as Core Node Property
• Policy Evaluation Engine #785 - Policy Evaluation Engine
• Sync Policy Infrastructure #784 - Sync Policy Infrastructure (also listed in Stream D)

Stream F: Backup & Recovery

Implement 3-2-1 backup strategy:

• Add export_to_json Tauri command
• Add import_from_json Tauri command
• Add backup_rocksdb command
• Implement GDPR export/anonymization

Stream G: Advanced Features (Future)

Deferred until core transition complete:

• Cloud workflow reactor
• Rate limiting for cloud executions
• Auto-optimization janitor

Dependencies

Stream A (Foundation) ✅ DONE
    ↓
Stream B (Data Model) ← START HERE
    ↓
Stream C (Security) ←→ Stream D (Sync)
    ↓                        ↓
Stream E (Governance) ←──────┘
    ↓
Stream F (Backup)
    ↓
Stream G (Advanced)

Success Criteria

• All spoke tables eliminated
• Single-query node retrieval (no N+1)
• Partial indexes maintain query performance
• All existing tests pass
• Custom entity types work without DDL
• Schema-as-Node enables playbook installation
• RLS enforced on SurrealDB Cloud

Estimated Effort

Stream	Effort	Priority	Status
A: Foundation	1-2 days	P0	✅ Done
B: Data Model	3-5 days	P0	Ready
C: Security	3-5 days	P1	Blocked by B
D: Sync	2-3 weeks	P1	Blocked by B
E: Governance	1 week	P1	Can parallel with D
F: Backup	2-3 days	P1	Blocked by B
G: Advanced	4+ weeks	P3	Future

Notes

• Pre-release system with zero users - breaking changes are acceptable
• No migration strategy needed - can reset database
• SurrealDB Cloud PoC should be done in separate repository to isolate from NodeSpace-specific issues

[malibio]

Starting Stream B implementation. Test baseline: Frontend 3431 passed, Rust 674 passed.