Skip to content

docs(rfc): Multi-Project Architecture design [AI-3027]#500

Draft
Matovidlo wants to merge 6 commits into
mainfrom
AI-3027-mpa-rfc
Draft

docs(rfc): Multi-Project Architecture design [AI-3027]#500
Matovidlo wants to merge 6 commits into
mainfrom
AI-3027-mpa-rfc

Conversation

@Matovidlo

Copy link
Copy Markdown
Contributor

Description

Linear: AI-3027

Change Type

  • Major (breaking changes, significant new features)
  • Minor (new features, enhancements, backward compatible)
  • Patch (bug fixes, small improvements, no new features)

Summary

RFC / design document for the Multi-Project Architecture (MPA) — a fundamental change to how the MCP server handles credentials and project context.

Problem: The current server is locked to a single Keboola project per session (one OAuth or Storage token at startup). This blocks Kai and enterprise users who need a single agent session to reason across multiple projects.

Solution: Two new credential types + an append-only session project registry + project_id on all project-scoped tools.

Key design decisions documented

  • Three token modes: Storage/OAuth (existing, unchanged) · PAT (single-project, never expires) · Programmatic token + refresh token (multi-project, org-wide access)
  • mcp.keboola.com is unaffected — OAuth users continue exactly as today. MPA is local/Kai only.
  • Append-only project registry: projects are added to a session but never removed, preventing agent confusion mid-conversation.
  • Explicit project_id on tools (chosen over middleware/header approach) — fully transparent to the LLM, reviewable in conversation history, natural for cross-project reasoning.
  • 24-hour idle refresh rule: programmatic token is refreshed automatically in the background; session expires after 24 h of inactivity.
  • Two new tools: list_accessible_projects and add_project_to_session.
  • Prompting changes: agent must discover + add projects before project-scoped tools work.

See docs/multi-project-architecture.md for the full RFC including flows, state design, access control matrix, and open questions.

Testing

RFC / design phase — no code changes. No tests applicable.

  • Tested with Cursor AI desktop (Streamable-HTTP transports)

Checklist

  • Self-review completed
  • Unit tests added/updated (if applicable)
  • Integration tests added/updated (if applicable)
  • Project version bumped according to the change type (if applicable)
  • Documentation updated (if applicable)

Documents the new multi-project architecture for the MCP server:
- Programmatic token + refresh token as new startup credentials
- PAT token for single-project headless/Kai use
- Append-only session project registry
- New tools: list_accessible_projects, add_project_to_session
- project_id argument on all project-scoped tools (vs middleware approach)
- Token refresh strategy (24h idle rule)
- Kai integration flow
- Deployment context table (mcp.keboola.com unaffected)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@linear

linear Bot commented May 4, 2026

Copy link
Copy Markdown

@Matovidlo

Copy link
Copy Markdown
Contributor Author

@claude review

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an RFC/design spec for Multi-Project Architecture (MPA) to enable a single MCP server session to work across multiple Keboola projects (primarily for Kai/local deployments), introducing new credential modes, an append-only project registry concept, and explicit project_id tool scoping.

Changes:

  • Documented token taxonomy (PAT vs programmatic token + refresh) and proposed auth flows for multi-project usage.
  • Proposed a session state shape including a project registry plus a background programmatic-token refresh loop.
  • Proposed tool-surface changes: new org-level tools and adding project_id to project-scoped tools.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/multi-project-architecture.md
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
@davidesner

Copy link
Copy Markdown
Contributor

@Matovidlo did you base it off #451? A lot of work has been done there that is reusable

@Matovidlo

Copy link
Copy Markdown
Contributor Author

@davidesner not really. This is just RFC, no implementation + diagrams. Once we confirm the design is correct I can run verification of the design against that PR

- add_project_to_session is purely in-memory; removed Management API /
  PAT-generation steps from Flow B diagram, tool spec, and Kai section
- Remove pat_token field from ProjectEntry (programmatic token used directly)
- Add security note: prefer env vars over CLI flags for token args
- Add stateful-session implementation note for Streamable-HTTP transport
- Replace HTTP 401/403 with ToolError throughout (tool errors are MCP-level,
  not HTTP status codes)
- Add project_id backward-compat note: optional when exactly one project active
- Clarify Open Question 1: Management API needed only for list_accessible_projects

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/multi-project-architecture.md
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
- Generalize sapi_client comment in ProjectEntry: credential is
  auth-mode-dependent (not always programmatic token)
- Add concrete project_id: str | None optional signature to from_project
  helper; show ambiguity ToolError and single-project defaulting logic
- Clarify Open Question 1: project list enumerated once at startup and
  cached; list_accessible_projects returns from cache (not re-fetched)
- Standardize "storage-token mode" → "Storage Token mode" in error message

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
…_query, get_flow_schema need project_id

All three tools use KeboolaClient.from_state() or call get_project_info() in
their current implementations:
- find_component_id: generates project-specific links via ProjectLinksManager
- docs_query: authenticates via project-scoped Storage API token
- get_flow_schema: gates conditional-flow schema by project-level feature flag

RFC updated: move all three to "needs project_id" table with explanation.
Add implementation note that a future refactor may decouple some from project
context. Also add Storage Token to the ProjectEntry sapi_client comment.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md
…nlyHint terminology

- Flow A: show project_id omitted in single-project mode (auto-resolved)
- Option 2 description: clarify project_id is optional at schema level (str | None = None),
  not required; runtime ambiguity checks in from_project enforce constraints
- AFTER example: update get_tables signature to project_id: str | None = None
- Refresh loop: clamp sleep to max(0, expires_at - now - 60s) to avoid
  negative or tight-loop on near-expiry token at startup
- Tool specs: rename Readonly to readOnlyHint to match ToolAnnotations(readOnlyHint=True)
  used in the actual codebase (keboola_mcp_server/mcp.py)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/multi-project-architecture.md Outdated
Comment thread docs/multi-project-architecture.md
Comment thread docs/multi-project-architecture.md
…n, add concurrency note

- Deployment Contexts table: normalize Multi-project column to boolean
  (❌ No (single project only)) for PAT/headless row
- add_project_to_session: readOnlyHint false — tool mutates server session
  state (appends to project_registry) even though no external API is called
- Token Refresh Strategy: document asyncio.Lock requirement for
  ProgrammaticTokenContext to prevent races between refresh loop and
  concurrent tool calls updating last_used_at / token fields

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/multi-project-architecture.md
Comment thread docs/multi-project-architecture.md
Comment thread docs/multi-project-architecture.md
Comment thread docs/multi-project-architecture.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants