[Initiative] Universal ScyllaDB Resource Table for CRD-Style Resources

[juliuskrah]

Summary

Define a universal, partitioned ScyllaDB resource table that can store arbitrary resource kinds in a Kubernetes-style layout. This enables the system to support new resource types without schema redesign while preserving efficient list/watch/query behavior.

In Scope

• Design a generic resources table partitioned by (kind, namespace) with name-based clustering
• Store stable identity, resourceVersion, labels, spec JSON, and status JSON in the table
• Define an identity mapping table for stable uid resolution across renames and recreates
• Specify query patterns for list, watch, and keyed lookups over the generic table
• Add migration and compatibility guidance for existing catalog-centric storage
• Include test coverage for multiple resource kinds and namespace partition behavior

Out of Scope

• Resource-specific business validation
• UI-specific query optimizations
• Git transport protocol changes
• A custom schema per resource kind

Acceptance Criteria

• A generic ScyllaDB resource table schema is documented and implemented
• The schema supports storing multiple kinds without separate tables per kind
• Identity mapping for stable uid resolution is defined and used
• List/watch query paths remain efficient for namespace-scoped lookups
• Migration guidance explains how catalog resources move onto the generic model
• Automated tests cover insertion, lookup, rename, and delete/recreate scenarios

Implementation Notes

The goal is to treat resources as first-class declarative objects rather than one-off catalog records. A partitioned schema gives the API a consistent storage contract for future CRD-like resource types while keeping reads predictable. The identity mapping table stays separate so lifecycle semantics remain stable even when a resource is renamed or recreated.

Dependencies

• Depends on: [Initiative] Datastore Backend: ScyllaDB with Migrations #102 (ScyllaDB backend), [Initiative] Stable Catalog Resource Identity and Rename-Aware UID Lifecycle #135 (stable catalog resource identity)
• Related: [Initiative] Controller Watch API with resourceVersion Resume #131 (watch API), [Initiative] Hydrated Catalog View in ScyllaDB with Lossless YAML Export #136 (hydrated catalog view), [Initiative] Object Lifecycle Versioning Contract for UID and resourceVersion #137 (lifecycle versioning contract)