Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion .github/workflows/ci-integration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,9 +113,23 @@ jobs:
go-version: '1.25.x'
cache-dependency-path: gitstore-api/go.sum

- name: Start ScyllaDB
run: |
docker run -d --name gitstore-scylla-test \
-p 9042:9042 \
scylladb/scylla:5.4 \
--developer-mode=1 --overprovisioned=1 --smp=1
timeout 120 sh -c 'until docker exec gitstore-scylla-test cqlsh -e "describe cluster"; do sleep 2; done'

- name: Run ScyllaDB contract tests
working-directory: ./gitstore-api
run: go test -tags scylla -v -timeout 10m ./tests/contract/datastore/... ./internal/datastore/scylla/...
env:
GITSTORE_TEST_SCYLLA_ADDR: 127.0.0.1:9042
run: go test -tags scylla -v -timeout 120s ./tests/contract/datastore/... ./internal/datastore/scylla/...

- name: Stop ScyllaDB
if: always()
run: docker rm -f gitstore-scylla-test || true

grpc-contract-test:
name: gRPC Contract Tests (testcontainers)
Expand Down
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ Auto-generated from all feature plans. Last updated: 2026-03-26
- Rust edition 2021, MSRV 1.82 (required by gix 0.83.0) + `gix 0.83.0` (replaces `git2 0.20.4`), `tokio 1.35`, `axum 0.8`, `tonic 0.14`, `tracing 0.1`, `anyhow 1.0` (007-migrate-gitoxide)
- Bare Git repositories on local filesystem (unchanged) (007-migrate-gitoxide)
- Rust edition 2021, MSRV 1.82 + `gix 0.83.0`, `gix-packetline` (compatible version), `gix-pack` (compatible version), `gix-protocol` (compatible version), `axum 0.8`, `tokio 1.35`, `tracing 0.1`, `tempfile 3.8` (dev) (008-remove-git-shellouts)
- Go 1.25 (`gitstore-api`) + `gqlgen v0.17.90`, `go-memdb v1.3.5`, `gocqlx/v3 v3.0.4` (ScyllaDB), `go-playground/validator/v10`, `go.uber.org/zap`, `google/uuid` (009-api-namespaces)
- `go-memdb` (development / in-memory backend) / ScyllaDB 5.x+ (production backend) — via the `datastore.Datastore` interface from feature 006 (009-api-namespaces)

- (001-git-backed-ecommerce)

Expand All @@ -21,9 +23,9 @@ Auto-generated from all feature plans. Last updated: 2026-03-26
: Follow standard conventions

## Recent Changes
- 009-api-namespaces: Added Go 1.25 (`gitstore-api`) + `gqlgen v0.17.90`, `go-memdb v1.3.5`, `gocqlx/v3 v3.0.4` (ScyllaDB), `go-playground/validator/v10`, `go.uber.org/zap`, `google/uuid`
- 008-remove-git-shellouts: Added Rust edition 2021, MSRV 1.82 + `gix 0.83.0`, `gix-packetline` (compatible version), `gix-pack` (compatible version), `gix-protocol` (compatible version), `axum 0.8`, `tokio 1.35`, `tracing 0.1`, `tempfile 3.8` (dev)
- 007-migrate-gitoxide: Added Rust edition 2021, MSRV 1.82 (required by gix 0.83.0) + `gix 0.83.0` (replaces `git2 0.20.4`), `tokio 1.35`, `axum 0.8`, `tonic 0.14`, `tracing 0.1`, `anyhow 1.0`
- 006-api-datastore-abstraction: Added Go 1.25 (`gitstore-api`)


<!-- MANUAL ADDITIONS START -->
Expand Down
101 changes: 95 additions & 6 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,15 +33,42 @@ The API gateway (`gitstore-api`) and the Git server (`gitstore-git-service`) com

Key environment variables:

| Service | Variable | Purpose |
|------------------------|----------------------|--------------------------------------------------------|
| `gitstore-api` | `GITSTORE_GIT__GRPC__URI` | gRPC address of git-service (e.g. `dns:///git-service:50051`) |
| `gitstore-api` | `GITSTORE_GIT__WS__URI` | WebSocket URL for catalogue-reload notifications |
| `gitstore-git-service` | `GITSTORE_GRPC__PORT` | Port the gRPC server binds on (default `50051`) |
| `gitstore-git-service` | `GITSTORE_GIT__DATA_DIR` | Path to the bare repository directory |
| Service | Variable | Purpose |
|------------------------|---------------------------|---------------------------------------------------------------|
| `gitstore-api` | `GITSTORE_GIT__GRPC__URI` | gRPC address of git-service (e.g. `dns:///git-service:50051`) |
| `gitstore-api` | `GITSTORE_GIT__WS__URI` | WebSocket URL for catalogue-reload notifications |
| `gitstore-git-service` | `GITSTORE_GRPC__PORT` | Port the gRPC server binds on (default `50051`) |
| `gitstore-git-service` | `GITSTORE_GIT__DATA_DIR` | Path to the bare repository directory |

These folders map directly to the control, storage, and distribution planes described below.

### Dynamic GraphQL Schema for CRD-Style Kinds

The platform supports CRD-style kinds, so GraphQL schema shape cannot be treated as fully static.

- `gitstore-api` should watch kind/definition registry updates from Git and trigger schema refresh.
- Runtime synthesis should translate JSON Schema-backed kind definitions into GraphQL object types and fields.
- Generated query roots should stay namespaced by domain (for example `query { catalog { product(id: "...") } }`).

Schema lifecycle should follow a safe publish pattern:

1. Build a candidate schema from current registry state.
2. Validate and wire resolvers.
3. Atomically publish if valid.
4. Keep the last known-good schema active on failure.

### Direct Synthesis vs Federation

For core kinds, prefer direct synthesis inside `gitstore-api` to reduce network hops and keep resolver behaviour predictable.

Federation is an optional path for externally owned integrations:

- External apps can expose independent subgraphs and extend shared entities.
- Composition uses federation ownership directives such as `@key`, `@extends`, and `@external`.
- Composition may run through an edge router/gateway when extension boundaries justify service isolation.

Use federation when an extension owns its own service boundary or datastore and must participate in cross-entity graph relationships. Keep core catalogue/resource kinds on direct synthesis by default.

### Git Engine — gitoxide (gix)

`gitstore-git-service` uses [gitoxide (`gix 0.83.0`)](https://github.com/Byron/gitoxide), a pure-Rust Git implementation, as its only Git library. The `git2` / libgit2 C binding was removed entirely in feature `007-migrate-gitoxide`.
Expand Down Expand Up @@ -307,3 +334,65 @@ graph TD
- Choose **Proposal 2** if strict release control and Git-native operational workflows are the primary priority.
- In both cases, Git remains authoritative and KV remains the read-optimised projection layer.

---

## Namespace Lifecycle Management (feature 009-api-namespaces)

Namespaces are the primary isolation boundary for repositories in GitStore. They are managed exclusively through the GraphQL API in `gitstore-api`; `gitstore-git-service` is unchanged (FR-011).

### Three Tiers

| Tier | Who can create | Owns repositories | Can have parent enterprise |
|----------------|-----------------------------|-------------------|----------------------------|
| `USER` | Any authenticated caller | Yes | No |
| `ORGANISATION` | Any authenticated caller | Yes | Optional |
| `ENTERPRISE` | Callers with `isAdmin` only | No | No |

### Global Identifier Uniqueness

Namespace identifiers are globally unique across all tiers. The same identifier cannot exist as both a user-space and an organisation namespace. Identifiers follow DNS label rules: lowercase alphanumeric + hyphens, 1–63 characters, no leading or trailing hyphen.

### Authorization Model

- **`isAdmin`** (JWT claim) is the elevated platform role. Callers with `isAdmin == true` may create enterprise namespaces and delete any namespace.
- **Ownership** for deletion is checked at query time via `CreatedBy == callerUsername || isAdmin`. No mutable ownership state is embedded in the JWT.

### API Surface

All namespace operations are GraphQL, consistent with the rest of the domain API. See `shared/schemas/namespace.graphqls` for the full contract.

```graphql
# Create a user namespace
mutation {
createNamespace(input: { identifier: "acme-corp", tier: USER }) {
namespace { id identifier tier createdAt createdBy }
}
}

# List all namespaces
query {
namespaces { id identifier tier createdBy }
}

# Get namespace by identifier
query {
namespace(identifier: "acme-corp") {
id identifier displayName tier parentEnterpriseId
createdAt createdBy updatedAt updatedBy
}
}

# Delete a namespace (owner or admin only)
mutation {
deleteNamespace(input: { identifier: "acme-corp" }) {
deletedIdentifier
}
}
```

### Deletion Guard

Deletion is blocked when the namespace contains repositories (enforced in the service layer). The guard is a no-op stub in this release (repositories table is out of scope); it will be enforced when the repository spec lands.

For quickstart examples and `curl`-based testing, see [`specs/009-api-namespaces/quickstart.md`](../specs/009-api-namespaces/quickstart.md).

11 changes: 10 additions & 1 deletion docs/developer-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -424,6 +424,15 @@ GITSTORE_LOG__LEVEL=info
docker compose -f compose.yml -f compose.scylla.yml up -d scylla scylla-init
```

Scylla-backed Go tests use an externally managed ScyllaDB instance. Start
ScyllaDB first, then point the tests at its CQL address:

```bash
cd gitstore-api
GITSTORE_TEST_SCYLLA_ADDR=127.0.0.1:9042 \
go test -tags scylla -v -timeout 10m ./tests/contract/datastore/... ./internal/datastore/scylla/...
```

### Go Licence Headers

All Go source files in this repository should include this header near the top of the file:
Expand Down Expand Up @@ -560,7 +569,7 @@ fn test_create_product_workflow() {
wscat -c ws://localhost:8080

# Check API logs
docker-compose logs api | grep websocket
docker compose logs api | grep websocket

# Manual cache invalidation
# TODO returns 404
Expand Down
9 changes: 4 additions & 5 deletions docs/implementation/006-api-datastore-abstraction.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,15 +86,14 @@ behavioural parity between backends:
# memdb (no external services)
go test ./tests/contract/datastore/...

# ScyllaDB (testcontainers — pulls image automatically)
go test -tags scylla -timeout 10m ./tests/contract/datastore/...
# ScyllaDB contract tests (requires an external ScyllaDB on 127.0.0.1:9042)
GITSTORE_TEST_SCYLLA_ADDR=127.0.0.1:9042 go test -tags scylla -timeout 10m ./tests/contract/datastore/...

# ScyllaDB backend unit tests + migration tests
go test -tags scylla -timeout 10m ./internal/datastore/scylla/...
GITSTORE_TEST_SCYLLA_ADDR=127.0.0.1:9042 go test -tags scylla -timeout 10m ./internal/datastore/scylla/...
```

To run against a pre-started ScyllaDB instance instead of testcontainers,
start the override compose stack at the repo root:
Start the override compose stack at the repo root before running ScyllaDB tests:

```bash
docker compose -f compose.yml -f compose.scylla.yml up -d scylla
Expand Down
2 changes: 1 addition & 1 deletion gitstore-admin/codegen.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
overwrite: true
schema: ../shared/schemas/*.graphql
schema: ../shared/schemas/*.graphqls
documents: 'src/graphql/**/*.graphql'
generates:
src/graphql/generated.ts:
Expand Down
1 change: 1 addition & 0 deletions gitstore-api/cmd/server/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ func main() {

// Create GraphQL resolver
resolver := graph.NewResolver(store, gitClient, logger.Log)
resolver.WithAuthMiddleware(authMiddleware)
schema := generated.NewExecutableSchema(generated.Config{Resolvers: resolver})
gqlServer := gqlhandler.NewDefaultServer(schema)

Expand Down
2 changes: 1 addition & 1 deletion gitstore-api/gqlgen.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Where are all the schema files located? globs are supported eg src/**/*.graphqls
schema:
- ../shared/schemas/*.graphql
- ../shared/schemas/*.graphqls
Comment thread
juliuskrah marked this conversation as resolved.

# Where should the generated server code go?
exec:
Expand Down
7 changes: 7 additions & 0 deletions gitstore-api/internal/datastore/datastore.go
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,13 @@ type Datastore interface {
UpdateCollection(ctx context.Context, c *Collection) error
DeleteCollection(ctx context.Context, id string) error

// Namespace operations
CreateNamespace(ctx context.Context, ns *Namespace) error
GetNamespace(ctx context.Context, id string) (*Namespace, error)
GetNamespaceByIdentifier(ctx context.Context, identifier string) (*Namespace, error)
ListNamespaces(ctx context.Context) ([]*Namespace, error)
DeleteNamespace(ctx context.Context, id string) error

// Lifecycle
Close() error
}
22 changes: 22 additions & 0 deletions gitstore-api/internal/datastore/entities.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,28 @@ package datastore

import "time"

// NamespaceTier is the enumeration of allowed namespace tiers.
type NamespaceTier string

const (
NamespaceTierUser NamespaceTier = "user"
NamespaceTierOrganisation NamespaceTier = "organisation"
NamespaceTierEnterprise NamespaceTier = "enterprise"
)

// Namespace is the primary isolation boundary for repositories.
type Namespace struct {
ID string
Identifier string
DisplayName string
Tier NamespaceTier
ParentEnterpriseID *string
CreatedAt time.Time
CreatedBy string
UpdatedAt time.Time
UpdatedBy string
}

// Product represents a sellable item in the catalogue.
type Product struct {
ID string
Expand Down
37 changes: 37 additions & 0 deletions gitstore-api/internal/datastore/instrumented.go
Original file line number Diff line number Diff line change
Expand Up @@ -181,6 +181,43 @@ func (d *InstrumentedDatastore) DeleteCollection(ctx context.Context, id string)
return err
}

// ── Namespace ─────────────────────────────────────────────────────────────

func (d *InstrumentedDatastore) CreateNamespace(ctx context.Context, ns *Namespace) error {
start := time.Now()
err := d.next.CreateNamespace(ctx, ns)
d.observe("CreateNamespace", start, err)
return err
}

func (d *InstrumentedDatastore) GetNamespace(ctx context.Context, id string) (*Namespace, error) {
start := time.Now()
v, err := d.next.GetNamespace(ctx, id)
d.observe("GetNamespace", start, err)
return v, err
}

func (d *InstrumentedDatastore) GetNamespaceByIdentifier(ctx context.Context, identifier string) (*Namespace, error) {
start := time.Now()
v, err := d.next.GetNamespaceByIdentifier(ctx, identifier)
d.observe("GetNamespaceByIdentifier", start, err)
return v, err
}

func (d *InstrumentedDatastore) ListNamespaces(ctx context.Context) ([]*Namespace, error) {
start := time.Now()
v, err := d.next.ListNamespaces(ctx)
d.observe("ListNamespaces", start, err)
return v, err
}

func (d *InstrumentedDatastore) DeleteNamespace(ctx context.Context, id string) error {
start := time.Now()
err := d.next.DeleteNamespace(ctx, id)
d.observe("DeleteNamespace", start, err)
return err
}

// ── Lifecycle ──────────────────────────────────────────────────────────────

func (d *InstrumentedDatastore) Close() error {
Expand Down
15 changes: 15 additions & 0 deletions gitstore-api/internal/datastore/instrumented_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,21 @@ func (s *stubDatastore) UpdateCollection(_ context.Context, _ *datastore.Collect
func (s *stubDatastore) DeleteCollection(_ context.Context, _ string) error {
return s.getProductErr
}
func (s *stubDatastore) CreateNamespace(_ context.Context, _ *datastore.Namespace) error {
return s.getProductErr
}
func (s *stubDatastore) GetNamespace(_ context.Context, _ string) (*datastore.Namespace, error) {
return nil, s.getProductErr
}
func (s *stubDatastore) GetNamespaceByIdentifier(_ context.Context, _ string) (*datastore.Namespace, error) {
return nil, s.getProductErr
}
func (s *stubDatastore) ListNamespaces(_ context.Context) ([]*datastore.Namespace, error) {
return nil, s.getProductErr
}
func (s *stubDatastore) DeleteNamespace(_ context.Context, _ string) error {
return s.getProductErr
}
func (s *stubDatastore) Close() error { return nil }

// newTestInstrumented creates an InstrumentedDatastore with an observer logger
Expand Down
Loading
Loading