Skip to content

optimize chat agent#5

Merged
larryro merged 3 commits into
mainfrom
optimize-chat-agent
Dec 4, 2025
Merged

optimize chat agent#5
larryro merged 3 commits into
mainfrom
optimize-chat-agent

Conversation

@larryro

@larryro larryro commented Dec 4, 2025

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

Bug Fixes

  • Improved customer search and filtering logic with enhanced query capabilities.

New Features

  • Added address field to customer records.
  • Enhanced agent data source selection to intelligently choose between search, database queries, and web lookups.

Chores

  • Simplified customer data structure by removing legacy analytics fields.
  • Removed three automated workflow defaults for new organizations.
  • Updated workflow documentation with clearer structural guidance.

✏️ Tip: You can customize this high-level summary in your review settings.

- Remove unused customer fields: phone, tags, totalSpent, orderCount,
  firstPurchaseAt, lastPurchaseAt, churned_at, notes
- Delete get_customer_stats.ts helper (no longer needed)
- Remove RAG sync workflows for products, website pages, and customers
  from default organization workflows
- Update customer_read_tool default fields and documentation
- Enhance chat agent prompt to prioritize database tools (customer_read,
  product_read, workflow_read) for counting/listing queries over rag_search
- Fix update_workflow_step_tool documentation to clarify that nextSteps
  must be at updates.nextSteps, not nested inside updates.config
- Update circuly sync workflow to map address fields correctly
- Remove redundant getCustomers and listByOrganization functions
- Extend queryCustomers to support all filtering options:
  - status/source as single value or array
  - locale filter (array)
  - searchTerm for name/email/externalId search
  - field projection support
- Use async iteration instead of .collect() for better efficiency
- Update read_customer_list helper to use queryCustomers
- Update public getCustomers to delegate to queryCustomers
Cast result.page to Doc<'customers'>[] to ensure proper TypeScript
typing for the customers array in the table component.
@coderabbitai

coderabbitai Bot commented Dec 4, 2025

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

This PR consolidates customer data management across the Convex backend by removing eight legacy customer fields (phone, tags, totalSpent, orderCount, firstSpent, lastPurchaseAt, churned_at, notes), adding an address field, and replacing three query functions (getCustomers, listByOrganization, getCustomerStats) with a unified queryCustomers function. It updates the agent instructions to employ data-source selection logic instead of a rag_search-first approach, removes three default RAG sync workflows, and modifies customer tool documentation to reflect the new field set. The Circuly sync workflow is updated to populate the address field.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

  • queryCustomers implementation (query_customers.ts): Rewritten pagination and filtering logic using manual cursor iteration with support for single/array-based status/source, searchTerm search, locale filtering, and field projection—requires careful validation of edge cases and cursor correctness.
  • Agent instruction overhaul (create_chat_agent.ts): Significant logic shift from rag_search-first to multi-source decision tree; impacts how agent selects tools and structures responses—needs verification that guidance aligns with tool capabilities.
  • Schema and API surface changes: Eight fields removed from multiple customer mutations/queries (createCustomer, updateCustomer, updateCustomers, bulkCreateCustomers) and three exported functions deleted—requires verification that no callers remain and migration impact is understood.
  • Workflow removals (save_default_workflows.ts): Three default workflows removed (Product RAG, Website Pages RAG, Customer RAG Sync)—needs confirmation these are not critical for existing tenants.
  • Type export consolidation: New QueryCustomersArgs/QueryCustomersReturns definitions introduced; verify all call sites (tool helpers, dashboard component) correctly use the new structure.

Possibly related PRs

  • tale-project/poc2#489: Modifies agent initialization logic and Convex tool registry concurrently with this PR's create_chat_agent and tool availability changes.
  • tale-project/poc2#486: Overlaps on customer model signature and query function consolidation (replacing getCustomers/listByOrganization paths).
  • refactor(agent_tools): consolidate tools with unified read operations #1: Directly related—both introduce customer_read tooling changes and remove legacy listByOrganization/getCustomers query paths alongside default field adjustments.

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
services/platform/convex/agent_tools/convex_tools/workflows/update_workflow_step_tool.ts (1)

147-178: Add error handling and validate updatedStep before reporting success.

The handler lacks try-catch error handling and unconditionally returns success: true even when updatedStep is null. This could mask failures such as invalid step IDs or mutation errors.

     handler: async (
       ctx,
       args,
     ): Promise<{
       success: boolean;
       message: string;
       step: Doc<'wfStepDefs'> | null;
     }> => {
       debugLog('update_workflow_step tool called', {
         stepRecordId: args.stepRecordId,
         updates: args.updates,
       });

-      const updatedStep = (await ctx.runMutation(
-        internal.wf_step_defs.updateStep,
-        {
-          stepRecordId: args.stepRecordId as Id<'wfStepDefs'>,
-          updates: args.updates as any,
-        },
-      )) as Doc<'wfStepDefs'> | null;
+      try {
+        const updatedStep = (await ctx.runMutation(
+          internal.wf_step_defs.updateStep,
+          {
+            stepRecordId: args.stepRecordId as Id<'wfStepDefs'>,
+            updates: args.updates as any,
+          },
+        )) as Doc<'wfStepDefs'> | null;

-      debugLog('update_workflow_step tool success', {
-        stepRecordId: args.stepRecordId,
-        updatedStep,
-      });
+        if (!updatedStep) {
+          debugLog('update_workflow_step tool failed - step not found', {
+            stepRecordId: args.stepRecordId,
+          });
+          return {
+            success: false,
+            message: `Step with ID ${args.stepRecordId} not found or update failed`,
+            step: null,
+          };
+        }

-      return {
-        success: true,
-        message: `Successfully updated step`,
-        step: updatedStep,
-      };
+        debugLog('update_workflow_step tool success', {
+          stepRecordId: args.stepRecordId,
+          updatedStep,
+        });
+
+        return {
+          success: true,
+          message: `Successfully updated step`,
+          step: updatedStep,
+        };
+      } catch (error) {
+        debugLog('update_workflow_step tool error', {
+          stepRecordId: args.stepRecordId,
+          error: error instanceof Error ? error.message : String(error),
+        });
+        return {
+          success: false,
+          message: `Failed to update step: ${error instanceof Error ? error.message : String(error)}`,
+          step: null,
+        };
+      }
     },
services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx (1)

50-60: Pagination is non-functional: cursor is hardcoded to null and totals are incorrect.

The current implementation has two issues:

  1. cursor: null means only the first page is ever fetched, regardless of currentPage
  2. pagination.total and totalPages are computed from the fetched page length, not the actual total count

For proper cursor-based pagination, the cursor from previous results must be passed on subsequent pages. The backend also doesn't provide a total count, making client-side pagination display misleading.

Consider either:

  • Implementing proper cursor tracking (store continueCursor and pass it for page navigation)
  • Adding a separate count query to get total records
  • Using infinite scroll pattern which aligns better with cursor-based pagination

Also applies to: 67-72

services/platform/convex/agent_tools/convex_tools/customers/helpers/types.ts (1)

23-30: Fix default fields mismatch: documentation claims all operations default to ['_id','name','email','status','source','locale'], but the list operation actually uses defaultListFields which excludes 'locale'.

The tool documentation (line 50 in customer_read_tool.ts) states the default fields as ['_id','name','email','status','source','locale'] for all operations, but:

  • get_by_id and get_by_email use defaultGetFields with 6 fields (includes 'locale')
  • list operation uses defaultListFields with 5 fields (excludes 'locale')

Either add 'locale' to defaultListFields to match the documented defaults and maintain consistency across operations, or update the documentation to reflect that list operations exclude 'locale'.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: ASSERTIVE

Plan: Pro (Legacy)

📥 Commits

Reviewing files that changed from the base of the PR and between ab75157 and bb95f0c.

⛔ Files ignored due to path filters (1)
  • services/platform/convex/_generated/api.d.ts is excluded by !**/_generated/**
📒 Files selected for processing (22)
  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx (2 hunks)
  • services/platform/convex/agent_tools/convex_tools/customers/customer_read_tool.ts (2 hunks)
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts (1 hunks)
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/types.ts (1 hunks)
  • services/platform/convex/agent_tools/convex_tools/workflows/update_workflow_step_tool.ts (2 hunks)
  • services/platform/convex/customers.ts (4 hunks)
  • services/platform/convex/lib/create_chat_agent.ts (7 hunks)
  • services/platform/convex/model/customers/bulk_create_customers.ts (0 hunks)
  • services/platform/convex/model/customers/create_customer.ts (2 hunks)
  • services/platform/convex/model/customers/create_customer_public.ts (0 hunks)
  • services/platform/convex/model/customers/get_customer_stats.ts (0 hunks)
  • services/platform/convex/model/customers/get_customers.ts (0 hunks)
  • services/platform/convex/model/customers/index.ts (0 hunks)
  • services/platform/convex/model/customers/list_by_organization.ts (0 hunks)
  • services/platform/convex/model/customers/query_customers.ts (3 hunks)
  • services/platform/convex/model/customers/types.ts (0 hunks)
  • services/platform/convex/model/customers/update_customer.ts (1 hunks)
  • services/platform/convex/model/customers/update_customers.ts (2 hunks)
  • services/platform/convex/model/organizations/save_default_workflows.ts (0 hunks)
  • services/platform/convex/predefined_workflows/circuly_sync_customers.ts (2 hunks)
  • services/platform/convex/schema.ts (0 hunks)
  • services/platform/convex/workflow/actions/customer/customer_action.ts (0 hunks)
💤 Files with no reviewable changes (10)
  • services/platform/convex/model/customers/get_customers.ts
  • services/platform/convex/model/customers/get_customer_stats.ts
  • services/platform/convex/workflow/actions/customer/customer_action.ts
  • services/platform/convex/model/customers/create_customer_public.ts
  • services/platform/convex/model/organizations/save_default_workflows.ts
  • services/platform/convex/model/customers/types.ts
  • services/platform/convex/schema.ts
  • services/platform/convex/model/customers/index.ts
  • services/platform/convex/model/customers/bulk_create_customers.ts
  • services/platform/convex/model/customers/list_by_organization.ts
🧰 Additional context used
📓 Path-based instructions (6)
**/*

📄 CodeRabbit inference engine (.cursor/rules/workspace_rules.mdc)

Use English only for ALL user-facing content including UI components, labels, buttons, dialogs, forms, toast messages, error messages, success messages, comments, documentation, README files, variable names, function names, and type names

Files:

  • services/platform/convex/model/customers/create_customer.ts
  • services/platform/convex/model/customers/update_customer.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/types.ts
  • services/platform/convex/model/customers/update_customers.ts
  • services/platform/convex/predefined_workflows/circuly_sync_customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/customer_read_tool.ts
  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
  • services/platform/convex/model/customers/query_customers.ts
  • services/platform/convex/lib/create_chat_agent.ts
  • services/platform/convex/customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
  • services/platform/convex/agent_tools/convex_tools/workflows/update_workflow_step_tool.ts
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (.cursor/rules/workspace_rules.mdc)

**/*.{ts,tsx,js,jsx}: Use Vercel AI SDK with OpenAI - import from 'ai' and '@ai-sdk/openai', never use raw OpenAI SDK or OpenRouter
Never hallucinate API keys - always use environment variables and existing .env configuration
Use camelCase for function names (e.g., getUserData)
Use SCREAMING_SNAKE_CASE for constants (e.g., API_BASE_URL, MAX_RETRIES)
Use feature flags with enums (TypeScript) or const objects (JavaScript) with UPPERCASE_WITH_UNDERSCORE naming
Implement error handling with try-catch pattern: check for result.error and display descriptive toast messages using result.error as title

Files:

  • services/platform/convex/model/customers/create_customer.ts
  • services/platform/convex/model/customers/update_customer.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/types.ts
  • services/platform/convex/model/customers/update_customers.ts
  • services/platform/convex/predefined_workflows/circuly_sync_customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/customer_read_tool.ts
  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
  • services/platform/convex/model/customers/query_customers.ts
  • services/platform/convex/lib/create_chat_agent.ts
  • services/platform/convex/customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
  • services/platform/convex/agent_tools/convex_tools/workflows/update_workflow_step_tool.ts
**/*.{ts,tsx}

📄 CodeRabbit inference engine (.cursor/rules/workspace_rules.mdc)

**/*.{ts,tsx}: Use kebab-case for file names (e.g., user-profile.tsx)
Use PascalCase for component names (e.g., UserProfile)
Use descriptive messages as toast title (never generic 'Error'), with optional description for additional context only
Follow component structure: 'use client' directive, imports, interface Props, hooks, effects, event handlers, then render
Prioritize data fetching methods in order: Server Actions (preferred), Route Handlers (when needed), Client-side (minimal use)
Use React.memo for expensive components to optimize performance
Use Next.js Image component for all images instead of native img tags
Use dynamic imports for code splitting

Files:

  • services/platform/convex/model/customers/create_customer.ts
  • services/platform/convex/model/customers/update_customer.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/types.ts
  • services/platform/convex/model/customers/update_customers.ts
  • services/platform/convex/predefined_workflows/circuly_sync_customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/customer_read_tool.ts
  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
  • services/platform/convex/model/customers/query_customers.ts
  • services/platform/convex/lib/create_chat_agent.ts
  • services/platform/convex/customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
  • services/platform/convex/agent_tools/convex_tools/workflows/update_workflow_step_tool.ts
**/*.{tsx,jsx}

📄 CodeRabbit inference engine (.cursor/rules/figma_rules.mdc)

**/*.{tsx,jsx}: Avoid specifying font-['Inter:Regular',_sans-serif] as it should be the default
Only specify font-family when using non-default fonts like font-['Inter:Medium',_sans-serif]
Ensure font-family matches font-weight (Inter:Regular with font-normal, Inter:Medium with font-medium)
Use leading-normal instead of leading-[normal] in Tailwind classes
Use standard font size classes instead of arbitrary values: text-[12px]text-xs, text-[14px]text-sm, text-[16px]text-base, text-[18px]text-lg, text-[20px]text-xl, text-[24px]text-2xl
Use semantic spacing classes: p-[4px]p-1, p-[8px]p-2, m-[4px]m-1, m-[8px]m-2
Convert pixel values to rem using the 16px base for width and height measurements: w-[278px]w-[17.375rem], h-[48px]h-[3rem], min-w-[120px]min-w-[7.5rem], max-w-[400px]max-w-[25rem]
NEVER use hardcoded colors like text-gray-500, bg-gray-100, border-gray-200; ALWAYS use design system semantic colors: text-foreground for primary text, text-muted-foreground for secondary text and icons, bg-background for main backgrounds, bg-muted for subtle backgrounds and hover states, border-border for borders
ALWAYS use the Table component instead of custom flex layouts; use Table, TableHeader, TableBody, TableRow, TableHead, TableCell components with proper column widths using rem units and semantic colors

Files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
**/app/**/*.tsx

📄 CodeRabbit inference engine (.cursor/rules/workspace_rules.mdc)

In Next.js App Router, use page.tsx as server components by default; use 'use client' only for interactions and state

Files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
services/*/convex/*.ts

📄 CodeRabbit inference engine (.cursor/rules/workspace_rules.mdc)

Thin wrapper API modules (like services/platform/convex/documents.ts) may export multiple Convex functions as wrappers that delegate to model helpers, but must not contain business logic and must only perform argument/return validation and delegation

Files:

  • services/platform/convex/customers.ts
🧠 Learnings (35)
📚 Learning: 2025-08-21T15:03:10.828Z
Learnt from: CR
Repo: talecorp/lanserhof PR: 0
File: .cursor/rules/supabase.mdc:0-0
Timestamp: 2025-08-21T15:03:10.828Z
Learning: Applies to supabase/types.ts : Do not edit `types.ts`; it is generated by the script

Applied to files:

  • services/platform/convex/agent_tools/convex_tools/customers/helpers/types.ts
  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : Use `ctx.db.patch` to shallow merge updates into an existing document. This method throws an error if the document does not exist.

Applied to files:

  • services/platform/convex/model/customers/update_customers.ts
📚 Learning: 2025-10-03T11:34:20.628Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-10-03T11:34:20.628Z
Learning: Applies to convex/**/*.{ts,js} : Use ctx.db.replace to fully replace a document; use ctx.db.patch for shallow updates

Applied to files:

  • services/platform/convex/model/customers/update_customers.ts
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : Be strict with types, particularly around document ids. If a function takes in an id for a document in the 'users' table, take in `Id<'users'>` rather than `string`.

Applied to files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-07-19T15:30:00.886Z
Learnt from: CR
Repo: talecorp/poc PR: 0
File: .cursor/rules/core-rules.mdc:0-0
Timestamp: 2025-07-19T15:30:00.886Z
Learning: Applies to **/*.{ts,tsx,js,jsx} : Add TypeScript types where missing and create reusable patterns for similar issues

Applied to files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-10-11T11:46:02.452Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursorrules:0-0
Timestamp: 2025-10-11T11:46:02.452Z
Learning: Applies to **/*.{ts,tsx,js} : Maintain type safety throughout the codebase

Applied to files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-07-20T08:40:41.255Z
Learnt from: CR
Repo: talecorp/poc PR: 0
File: .cursor/rules/supabase.mdc:0-0
Timestamp: 2025-07-20T08:40:41.255Z
Learning: Applies to supabase/types.ts : `types.ts` was generated by the script. Never edit it manually

Applied to files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-07-03T08:43:49.346Z
Learnt from: CR
Repo: talecorp/poc PR: 0
File: .cursor/rules/next-best-practice.mdc:0-0
Timestamp: 2025-07-03T08:43:49.346Z
Learning: Applies to **/*.{ts,tsx} : Define proper types using TypeScript interfaces and types

Applied to files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-10-03T11:34:20.628Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-10-03T11:34:20.628Z
Learning: Applies to convex/**/*.ts : Use Id helper type from ./_generated/dataModel to type document ids (e.g., Id<'users'>)

Applied to files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-11-30T03:53:00.316Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-11-30T03:53:00.316Z
Learning: Applies to convex/**/*.ts : Be strict with types, particularly around IDs of documents; use `Id<'tableName'>` rather than `string` for function arguments and returns

Applied to files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-10-01T17:12:39.508Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursor/rules/figma_rules.mdc:0-0
Timestamp: 2025-10-01T17:12:39.508Z
Learning: Applies to **/*.tsx : Always use the Table component instead of custom flex layouts for tabular data

Applied to files:

  • services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx
📚 Learning: 2025-11-30T03:53:00.316Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-11-30T03:53:00.316Z
Learning: Applies to convex/**/*.ts : Always use the new Convex function syntax with `query`, `mutation`, `internalQuery`, `internalMutation`, `action`, or `internalAction` with explicit `args`, `returns`, and `handler` properties

Applied to files:

  • services/platform/convex/model/customers/query_customers.ts
  • services/platform/convex/customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : Use the `paginationOptsValidator` from `convex/server` for paginated queries with `numItems` and `cursor` parameters. Queries ending in `.paginate()` return objects with `page`, `isDone`, and `continueCursor` properties.

Applied to files:

  • services/platform/convex/model/customers/query_customers.ts
  • services/platform/convex/customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
📚 Learning: 2025-11-30T03:53:00.316Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-11-30T03:53:00.316Z
Learning: Applies to convex/**/*.ts : Define pagination using `paginationOptsValidator` with `numItems` and `cursor` properties, and use `.paginate()` on queries which returns objects with `page`, `isDone`, and `continueCursor` properties

Applied to files:

  • services/platform/convex/model/customers/query_customers.ts
  • services/platform/convex/customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
📚 Learning: 2025-07-20T08:40:24.693Z
Learnt from: CR
Repo: talecorp/poc PR: 0
File: .cursor/rules/ai.mdc:0-0
Timestamp: 2025-07-20T08:40:24.693Z
Learning: Applies to **/lib/**/*.ts : For text generation, use the `generateObject` or similar functions from the AI SDK

Applied to files:

  • services/platform/convex/lib/create_chat_agent.ts
📚 Learning: 2025-07-20T08:40:24.693Z
Learnt from: CR
Repo: talecorp/poc PR: 0
File: .cursor/rules/ai.mdc:0-0
Timestamp: 2025-07-20T08:40:24.693Z
Learning: Applies to **/app/api/**/*.ts : For text generation, use the `generateObject` or similar functions from the AI SDK

Applied to files:

  • services/platform/convex/lib/create_chat_agent.ts
📚 Learning: 2025-07-20T08:40:24.693Z
Learnt from: CR
Repo: talecorp/poc PR: 0
File: .cursor/rules/ai.mdc:0-0
Timestamp: 2025-07-20T08:40:24.693Z
Learning: Applies to **/actions/*.ts : For text generation, use the `generateObject` or similar functions from the AI SDK

Applied to files:

  • services/platform/convex/lib/create_chat_agent.ts
📚 Learning: 2025-11-30T12:29:39.745Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/workspace_rules.mdc:0-0
Timestamp: 2025-11-30T12:29:39.745Z
Learning: Applies to services/**/convex/*.ts : Thin wrapper API modules in services may export multiple Convex functions as thin wrappers that delegate to model helpers, must use snake_case file names and camelCase export names, and must not contain business logic

Applied to files:

  • services/platform/convex/lib/create_chat_agent.ts
📚 Learning: 2025-11-30T03:53:00.316Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-11-30T03:53:00.316Z
Learning: Applies to convex/**/!(model)/*.ts : Exception: Thin wrapper API modules (e.g., `services/platform/convex/documents.ts`) may export multiple Convex functions if they only validate args/returns and delegate to the model layer; must use snake_case for file names and camelCase for function names

Applied to files:

  • services/platform/convex/lib/create_chat_agent.ts
📚 Learning: 2025-10-03T11:34:20.628Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-10-03T11:34:20.628Z
Learning: Applies to convex/**/*.{ts,js} : Always use the new Convex function syntax (query/mutation/action with args/returns/handler)

Applied to files:

  • services/platform/convex/customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
📚 Learning: 2025-12-02T08:13:51.379Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/workspace_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:51.379Z
Learning: Applies to services/*/convex/*.ts : Thin wrapper API modules (like `services/platform/convex/documents.ts`) may export multiple Convex functions as wrappers that delegate to model helpers, but must not contain business logic and must only perform argument/return validation and delegation

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : Thoughtfully organize files with public query, mutation, or action functions within the `convex/` directory following file-based routing conventions.

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-11-30T03:53:00.316Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-11-30T03:53:00.316Z
Learning: Applies to convex/**/*.ts : Always include argument and return validators for all Convex functions (query, internalQuery, mutation, internalMutation, action, internalAction)

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : Use `query`, `mutation`, and `action` to register public functions that are part of the public API. Do NOT use these for sensitive internal functions.

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : Try to use as few calls from actions to queries and mutations as possible, as they are transactions and splitting logic introduces race condition risks.

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : ALWAYS include argument and return validators for all Convex functions, including `query`, `internalQuery`, `mutation`, `internalMutation`, `action`, and `internalAction`. If a function doesn't return anything, include `returns: v.null()`.

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-11-30T03:53:00.316Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-11-30T03:53:00.316Z
Learning: Applies to convex/**/*.ts : Use the `internal` object from `convex/_generated/api.ts` to call internal functions registered with `internalQuery`, `internalMutation`, or `internalAction`

Applied to files:

  • services/platform/convex/customers.ts
  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
📚 Learning: 2025-11-30T03:53:00.316Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-11-30T03:53:00.316Z
Learning: Applies to convex/**/*.ts : Try to use as few calls from actions to queries and mutations as possible to avoid race conditions, since queries and mutations are transactions

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-10-03T11:34:20.628Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-10-03T11:34:20.628Z
Learning: Applies to convex/**/*.{ts,js} : Register internal functions with internalQuery, internalMutation, and internalAction (imported from ./_generated/server)

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-11-30T03:53:00.316Z
Learnt from: CR
Repo: tale-project/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-11-30T03:53:00.316Z
Learning: Applies to convex/**/*.ts : Use the `api` object from `convex/_generated/api.ts` to call public functions registered with `query`, `mutation`, or `action`

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-10-03T11:34:20.628Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-10-03T11:34:20.628Z
Learning: Applies to convex/**/*.{ts,js} : Define paginated queries using paginationOptsValidator and .paginate(opts)

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : Use `internalQuery`, `internalMutation`, and `internalAction` to register internal (private) functions that can only be called by other Convex functions and are not exposed to the public API.

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-10-03T11:34:20.628Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-10-03T11:34:20.628Z
Learning: Applies to convex/**/*.{ts,js} : Register public functions with query, mutation, and action; do not use these for sensitive internal logic

Applied to files:

  • services/platform/convex/customers.ts
📚 Learning: 2025-12-02T08:13:24.266Z
Learnt from: CR
Repo: tale-project/tale PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-12-02T08:13:24.266Z
Learning: Applies to convex/**/*.{ts,tsx} : Use `ctx.runQuery` to call a query from a query, mutation, or action.

Applied to files:

  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
📚 Learning: 2025-10-03T11:34:20.628Z
Learnt from: CR
Repo: talecorp/poc2 PR: 0
File: .cursor/rules/convex_rules.mdc:0-0
Timestamp: 2025-10-03T11:34:20.628Z
Learning: Applies to convex/**/*.{ts,js} : Use ctx.runQuery to call queries, ctx.runMutation for mutations, and ctx.runAction for actions

Applied to files:

  • services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts
🧬 Code graph analysis (2)
services/platform/convex/model/customers/query_customers.ts (1)
services/platform/convex/model/customers/index.ts (1)
  • QueryCustomersArgs (18-18)
services/platform/convex/customers.ts (3)
services/platform/convex/model/customers/types.ts (3)
  • customerAddressValidator (33-39)
  • customerStatusValidator (15-19)
  • customerSourceValidator (24-28)
services/platform/convex/model/customers/query_customers.ts (1)
  • queryCustomers (49-165)
services/platform/convex/model/customers/index.ts (1)
  • queryCustomers (16-16)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Build services/platform
🔇 Additional comments (22)
services/platform/convex/agent_tools/convex_tools/workflows/update_workflow_step_tool.ts (2)

22-77: Documentation improvements look good.

The added structure rules with correct/wrong examples provide clear guidance for AI agents. The explicit separation of config and nextSteps fields and the requirement for type in ACTION steps are well documented.


138-143: Documentation for nextSteps field is clear and helpful.

The emphasis on nextSteps being a TOP-LEVEL field in updates (not inside config) aligns well with the CRITICAL STRUCTURE RULES in the main description.

services/platform/convex/lib/create_chat_agent.ts (5)

6-6: Clean type-only import.

Good use of the type modifier for importing ToolName, which ensures it's erased at runtime and signals that only the type is needed.


76-94: Well-designed data source selection framework.

The clear distinction between rag_search (semantic/vector similarity) and database tools (structured queries) is excellent. The explicit RULE on line 94 provides unambiguous guidance for counting/listing operations, which should prevent the agent from returning incomplete results.


168-183: Source of truth correctly updated.

The "NO HALLUCINATIONS / SOURCE OF TRUTH" section appropriately includes database tool results (customer_read, product_read, workflow_read) as allowed sources, maintaining consistency with the new data source selection framework introduced earlier.


191-193: Clarification examples align with tool guidance.

The updated example on line 192 asking whether the user wants to "count, list, search, or get a specific customer" directly reflects the operations supported by the new database tools, providing consistent guidance throughout the instructions.


32-41: Good addition of database query tools.

Adding customer_read, product_read, and workflow_read to the default tool set aligns well with the documented instructions that guide the agent to use database tools for counting and listing operations instead of relying solely on rag_search.

services/platform/convex/model/customers/query_customers.ts (2)

86-95: Source filter silently excludes customers with null/undefined source.

When args.source is provided, customers without a source value are filtered out. This may be intentional, but if users expect to include records with missing source values, this behavior could be unexpected. Consider whether this is the desired behavior.


137-156: Field projection implementation looks correct.

The projection logic properly handles the fields array and returns a narrowed object type. The use of args.fields! is safe here since it's within the if (args.fields && args.fields.length > 0) guard.

services/platform/app/(app)/dashboard/[id]/(knowledge)/customers/customers-table.tsx (1)

25-25: Type assertion is appropriate given the API contract.

The Doc<'customers'>[] cast is necessary because QueryCustomersResult.page is a union type. Since the UI doesn't pass fields to the query, full documents are always returned, making this assertion safe.

Also applies to: 66-66

services/platform/convex/agent_tools/convex_tools/customers/helpers/read_customer_list.ts (1)

23-35: Clean migration to the unified queryCustomers function.

The explicit result type annotation correctly includes count to match the new QueryCustomersResult interface. The rest of the function logic remains intact.

services/platform/convex/model/customers/create_customer.ts (2)

14-20: Address structure follows standard conventions.

The address object with optional street, city, state, country, and postalCode fields is well-structured. All fields being optional allows partial address information.


29-39: Schema already includes the address field.

The address field is properly defined in the customers table schema as an optional nested object containing street, city, state, and country properties. No further action needed.

services/platform/convex/predefined_workflows/circuly_sync_customers.ts (1)

147-152: Potential runtime error if address.shipping is null/undefined.

If a Circuly customer doesn't have a shipping address (loop.item.address.shipping is null), the template expressions may fail or produce unexpected values. Consider adding defensive handling in the workflow executor or documenting that shipping address is expected to exist.

Also applies to: 181-186

services/platform/convex/model/customers/update_customers.ts (2)

33-39: Address field added to update interface.

The address field has been correctly added to the updates interface with appropriate nested optional fields.


97-98: Consider documenting the shallow merge behavior for the address field.

The address field is assigned directly, which means ctx.db.patch will perform a shallow merge and replace the entire address object. This differs from the metadata handling (lines 101-134) which uses lodash for deep merging.

If a user wants to update only the city, they must provide the complete address object (including street, state, etc.), otherwise those fields will be lost.

Please confirm this shallow merge behavior is intentional for address updates. Consider adding a comment to document this behavior for future maintainers, or implement merge logic similar to metadata if partial address updates should be supported.

Based on learnings, ctx.db.patch performs shallow merge on nested objects.

services/platform/convex/model/customers/update_customer.ts (2)

50-50: LGTM - Minor formatting improvement.

Error message formatting consolidated to a single line for consistency.


16-22: Same shallow merge behavior applies to address field here.

Similar to update_customers.ts, the address field will be shallow-merged via ctx.db.patch (line 80). Users must provide the complete address object when updating any address field to avoid losing other fields.

Ensure this behavior is consistent with your design intent and is documented for API consumers.

Based on learnings, ctx.db.patch performs shallow merge on nested objects.

services/platform/convex/customers.ts (3)

83-105: Excellent refactoring - shared query definitions reduce duplication.

The extraction of queryCustomersArgs and queryCustomersReturns as shared constants is a good practice that ensures consistency between internal and public query operations.


36-36: Consistent address field integration across all customer operations.

The address field has been properly added to all relevant operations using customerAddressValidator consistently. The validator structure aligns with the model layer changes.

Also applies to: 133-133, 203-203, 237-237


169-176: Well-structured consolidation of customer query operations.

The refactored getCustomers function properly delegates to CustomersModel.queryCustomers and uses the shared args/returns definitions, maintaining the thin wrapper pattern while providing RLS protection.

As per coding guidelines, thin wrapper API modules should only perform argument/return validation and delegation without business logic.

services/platform/convex/agent_tools/convex_tools/customers/customer_read_tool.ts (1)

51-52: Updated field and best-practices documentation looks consistent and helpful.

The fields default description, new address field explanation, and the stronger warning about large metadata objects give clearer guidance to the agent while staying aligned with the current field set. No further changes needed here.

Also applies to: 92-103

Comment on lines +24 to +34
type CustomerStatus = 'active' | 'churned' | 'potential';
type CustomerSource = 'manual_import' | 'file_upload' | 'circuly';

export interface QueryCustomersArgs {
organizationId: string;
externalId?: string | number;
status?: 'active' | 'churned' | 'potential';
source?: 'manual_import' | 'file_upload' | 'circuly';
status?: CustomerStatus | CustomerStatus[];
source?: CustomerSource | string[];
locale?: string[];
searchTerm?: string;
fields?: string[];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Type mismatch between CustomerSource type and source argument.

The CustomerSource type is defined with specific literals, but source argument accepts CustomerSource | string[]. This creates an inconsistency where a single value must be CustomerSource but array values can be arbitrary strings. Consider aligning the types:

-type CustomerSource = 'manual_import' | 'file_upload' | 'circuly';
+export type CustomerSource = 'manual_import' | 'file_upload' | 'circuly';

 export interface QueryCustomersArgs {
   organizationId: string;
   externalId?: string | number;
   status?: CustomerStatus | CustomerStatus[];
-  source?: CustomerSource | string[];
+  source?: CustomerSource | CustomerSource[];
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
type CustomerStatus = 'active' | 'churned' | 'potential';
type CustomerSource = 'manual_import' | 'file_upload' | 'circuly';
export interface QueryCustomersArgs {
organizationId: string;
externalId?: string | number;
status?: 'active' | 'churned' | 'potential';
source?: 'manual_import' | 'file_upload' | 'circuly';
status?: CustomerStatus | CustomerStatus[];
source?: CustomerSource | string[];
locale?: string[];
searchTerm?: string;
fields?: string[];
type CustomerStatus = 'active' | 'churned' | 'potential';
export type CustomerSource = 'manual_import' | 'file_upload' | 'circuly';
export interface QueryCustomersArgs {
organizationId: string;
externalId?: string | number;
status?: CustomerStatus | CustomerStatus[];
source?: CustomerSource | CustomerSource[];
locale?: string[];
searchTerm?: string;
fields?: string[];
🤖 Prompt for AI Agents
In services/platform/convex/model/customers/query_customers.ts around lines
24-34, the source parameter accepts CustomerSource | string[] which allows
arbitrary strings only for arrays; change the type to be consistent by using
source?: CustomerSource | CustomerSource[] so both single and array values are
restricted to the CustomerSource union, and update any callsites/types/tests
that passed string[] to use the stricter CustomerSource[] or map/validate inputs
accordingly.

Comment on lines +56 to +62
// Use by_organizationId index and filter in the loop
const query = ctx.db
.query('customers')
.withIndex('by_organizationId', (q) =>
q.eq('organizationId', args.organizationId),
)
.order('desc');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Consider documenting performance characteristics of manual cursor-based pagination.

The implementation iterates through all records from the beginning until finding the cursor position, which can be O(n) where n is the offset position. For large datasets, this could be slow. Convex's built-in .paginate() method handles this more efficiently.

Based on learnings, Convex provides paginationOptsValidator with built-in .paginate() that returns page, isDone, and continueCursor. Consider whether the custom filtering requirements justify the manual pagination approach, or if a hybrid using .paginate() with post-filtering might work.

Comment on lines +130 to +135
// Check if we have enough items
if (customers.length >= numItems) {
hasMore = true;
break;
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

hasMore flag may be inaccurate.

Setting hasMore = true when customers.length >= numItems assumes there are more records, but breaking immediately after collecting numItems means you haven't checked if there's actually a next record. If exactly numItems records exist and match, hasMore will incorrectly be true.

A more accurate approach is to fetch one extra item to confirm more data exists:

-    // Check if we have enough items
-    if (customers.length >= numItems) {
-      hasMore = true;
-      break;
-    }
+    // Check if we have enough items (fetch numItems + 1 to detect hasMore)
+    if (customers.length > numItems) {
+      hasMore = true;
+      customers.pop(); // Remove the extra item
+      break;
+    }

This requires changing the loop condition to collect numItems + 1 items before breaking.

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents
In services/platform/convex/model/customers/query_customers.ts around lines
130-135, the code sets hasMore = true as soon as customers.length >= numItems
then breaks, which can incorrectly signal more data when exactly numItems exist;
change the collection logic to fetch up to numItems + 1 items, set hasMore =
customers.length > numItems, and then truncate the returned customers array to
numItems before returning so hasMore accurately reflects whether an extra item
exists beyond the requested page.

Comment on lines +147 to +152
address: {
street: '{{loop.item.address.shipping.street}}',
city: '{{loop.item.address.shipping.city}}',
country: '{{loop.item.address.shipping.country}}',
postalCode: '{{loop.item.address.shipping.postal_code}}',
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Missing state field in address mapping.

The CreateCustomerArgs.address interface includes a state field, but the Circuly sync doesn't map it. If Circuly provides state information (possibly as loop.item.address.shipping.state or region), consider adding it:

 address: {
   street: '{{loop.item.address.shipping.street}}',
   city: '{{loop.item.address.shipping.city}}',
+  state: '{{loop.item.address.shipping.state}}',
   country: '{{loop.item.address.shipping.country}}',
   postalCode: '{{loop.item.address.shipping.postal_code}}',
 },

Also applies to: 181-186

🤖 Prompt for AI Agents
In services/platform/convex/predefined_workflows/circuly_sync_customers.ts
around lines 147-152 and similarly at 181-186 the address mapping is missing the
required CreateCustomerArgs.address.state field; add a state property to the
mapped address, pulling from the Circuly payload (e.g.
loop.item.address.shipping.state or loop.item.address.shipping.region) and fall
back to undefined or an empty string if neither exists so the object conforms to
the CreateCustomerArgs.address interface.

@larryro larryro merged commit 0252776 into main Dec 4, 2025
2 checks passed
@larryro larryro deleted the optimize-chat-agent branch December 4, 2025 12:02
larryro added a commit that referenced this pull request Dec 30, 2025
…lidate

- repairObject now recursively processes array elements to repair corrupted
  keys inside nested objects within arrays
- validateObject now recursively validates array elements to catch control
  characters in nested object keys
- Added biome-ignore comments for intentional control character regex patterns
- Added camelCase normalization for repaired field names (e.g., userprompt -> userPrompt)

Addresses CodeRabbit review comments #5, #6, and #7.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
larryro added a commit that referenced this pull request Dec 30, 2025
…lidate

- repairObject now recursively processes array elements to repair corrupted
  keys inside nested objects within arrays
- validateObject now recursively validates array elements to catch control
  characters in nested object keys
- Added biome-ignore comments for intentional control character regex patterns
- Added camelCase normalization for repaired field names (e.g., userprompt -> userPrompt)

Addresses CodeRabbit review comments #5, #6, and #7.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
yannickmonney pushed a commit that referenced this pull request Apr 8, 2026
larryro added a commit that referenced this pull request May 7, 2026
…cies

Bundle of round-2-confirmed cross-tenant fixes plus the dead-code
delete of the semantic LLM response cache.

POLICY_TYPES drift (W6 #5)
- lib/shared/schemas/governance.ts now includes
  'data_classification_notice' to match the Convex enum, killing the
  `as const` cast at use-data-classification-notice.ts:50.

documents/compare_documents.ts (W6 #8)
- Convex `_storage` is a global namespace; org membership alone was
  not enough to gate `ctx.storage.getUrl`. Adds a JOIN through
  fileMetadata via the new internal query verifyStorageIdsBelongToOrg
  to confirm both `baseStorageId` and `comparisonStorageId` are owned
  by the caller's org. Refuses with a clear error otherwise. Pattern
  copied from agent_tools/documents/helpers/retrieve_document.ts.

file_metadata/actions.ts::checkFileRagStatuses (W6 #9)
- Was an unauthenticated public action that could flip any org's
  fileMetadata.ragStatus to `failed` via expireStaleRagQueue (DoS,
  pre-existing on `main`). Now requires `getAuthUser` and filters
  storageIds to ones owned by an org the caller is a member of via
  the new file_metadata.internal_queries.filterStorageIdsByCallerOrg.

governance/queries.ts (W6 #11)
- getPolicy + listPolicies now apply a member-readable allow-list
  (data_classification_notice, feature_flags, pii_config,
  chat_filter, personalization, upload_policy, default_models). All
  other types — login_policy.trustedProxies, password_policy,
  two_factor_policy, model_access.rules, budgets, retention_policy,
  moderation_provider.endpoint, system_prompt — are admin-only.
  listPolicies silently filters those out for non-admins.

semantic LLM response cache — DELETE (W6 #12 + #13)
- Round-2 v05 confirmed the lookup is structurally cross-tenant
  (filters only on agent_name, model, expires_at, similarity; ignores
  user_id / organization_id even though they're stored). The platform
  helpers `lookupSemanticCache` / `storeSemanticCacheAsync` had ZERO
  callers in the monorepo, the FastAPI router was mounted but
  unreachable from platform — a latent foot-gun primed for the next
  dev to wire up unaware. Deletes:

  - services/platform/convex/lib/response_cache/semantic_cache.ts
  - services/platform/convex/lib/response_cache/internal_actions.ts
  - services/rag/app/routers/llm_cache.py
  - services/rag/app/services/llm_response_cache.py

  Plus the corresponding imports in routers/__init__.py, main.py,
  rag_service.py. Also removes the two empty-catch violations in
  semantic_cache.ts (no longer applicable).

The exact-key Convex `lib/response_cache/{internal_mutations,
internal_queries}.ts` cache stays — it is the actually-wired one and
is correctly org-scoped.
larryro added a commit that referenced this pull request May 7, 2026
…eout

Round-2 v15 confirmed: /config unauthenticated, /openapi.json + /docs +
/redoc unauthenticated, RAG container ran as root, default token baked
into image ENV, strict-mode env name diverged across the wire,
non-constant-time token compare, plus three SSRF-guard gaps.

services/rag/app/auth.py
- W7 #3: hmac.compare_digest replaces == on the bearer compare. Removes
  the dead-code EXEMPT_PATHS frozenset.

services/rag/app/routers/health.py
- W7 #1: split into public_router (`/`, `/health`) and protected_router
  (`/config`). main.py mounts the protected one under
  Depends(verify_internal_token). Old `router` re-export stays for
  backwards compat.

services/rag/app/main.py
- W7 #2: docs_url / redoc_url / openapi_url are None outside debug.
- W7 #4: CORS allow_credentials flipped to False (bearer rides
  Authorization, never cookies).
- W7 #1 wiring: mount health-public + health-protected separately.

services/rag/app/config.py
- W7 #8: require_custom_internal_token accepts BOTH
  RAG_REQUIRE_CUSTOM_INTERNAL_TOKEN and TALE_REQUIRE_CUSTOM_RAG_TOKEN
  via pydantic AliasChoices.

services/rag/Dockerfile + services/convex/Dockerfile
- W7 #5: RAG container runs as non-root (uid:gid 1001:1001 `app`).
  RAG ingests untrusted PDFs/DOCX through native parsers; biggest
  blast radius in the stack, now hardened.
- W7 #6: removed RAG_INTERNAL_TOKEN=tale-rag-dev-only ENV bake from
  both runtime + scratch-squash stages and the matching bake in
  services/convex/Dockerfile. Operators MUST supply via env / compose
  / k8s secret.

services/platform/convex/lib/helpers/rag_config.ts
- W7 #9 F1: `redirect: 'manual'` on every ragFetch.
- W7 #9 F2: added fc00::/7 (IPv6 ULA) to v6 blocklist (AWS IPv6 IMDSv2).
- W7 #9 F3: strip trailing `.` before hostname blocklist lookup.
- W7 #9 F4: re-validate URL per ragFetch invocation (DNS rebinding +
  env rotation mitigation).
- W7 #9 F9: deleted path.startsWith('http') override branch (future-
  bypass foot-gun).

services/platform/convex/agent_tools/rag/helpers/fetch_document_chunks.ts
- W7 #10: pass timeoutMs=60_000 (default 10s was a regression).
- Plus MAX_ITERATIONS=30 cap and "cursor did not advance" break to
  defend against an adversarial RAG response.
larryro added a commit that referenced this pull request May 8, 2026
…cies

Bundle of round-2-confirmed cross-tenant fixes plus the dead-code
delete of the semantic LLM response cache.

POLICY_TYPES drift (W6 #5)
- lib/shared/schemas/governance.ts now includes
  'data_classification_notice' to match the Convex enum, killing the
  `as const` cast at use-data-classification-notice.ts:50.

documents/compare_documents.ts (W6 #8)
- Convex `_storage` is a global namespace; org membership alone was
  not enough to gate `ctx.storage.getUrl`. Adds a JOIN through
  fileMetadata via the new internal query verifyStorageIdsBelongToOrg
  to confirm both `baseStorageId` and `comparisonStorageId` are owned
  by the caller's org. Refuses with a clear error otherwise. Pattern
  copied from agent_tools/documents/helpers/retrieve_document.ts.

file_metadata/actions.ts::checkFileRagStatuses (W6 #9)
- Was an unauthenticated public action that could flip any org's
  fileMetadata.ragStatus to `failed` via expireStaleRagQueue (DoS,
  pre-existing on `main`). Now requires `getAuthUser` and filters
  storageIds to ones owned by an org the caller is a member of via
  the new file_metadata.internal_queries.filterStorageIdsByCallerOrg.

governance/queries.ts (W6 #11)
- getPolicy + listPolicies now apply a member-readable allow-list
  (data_classification_notice, feature_flags, pii_config,
  chat_filter, personalization, upload_policy, default_models). All
  other types — login_policy.trustedProxies, password_policy,
  two_factor_policy, model_access.rules, budgets, retention_policy,
  moderation_provider.endpoint, system_prompt — are admin-only.
  listPolicies silently filters those out for non-admins.

semantic LLM response cache — DELETE (W6 #12 + #13)
- Round-2 v05 confirmed the lookup is structurally cross-tenant
  (filters only on agent_name, model, expires_at, similarity; ignores
  user_id / organization_id even though they're stored). The platform
  helpers `lookupSemanticCache` / `storeSemanticCacheAsync` had ZERO
  callers in the monorepo, the FastAPI router was mounted but
  unreachable from platform — a latent foot-gun primed for the next
  dev to wire up unaware. Deletes:

  - services/platform/convex/lib/response_cache/semantic_cache.ts
  - services/platform/convex/lib/response_cache/internal_actions.ts
  - services/rag/app/routers/llm_cache.py
  - services/rag/app/services/llm_response_cache.py

  Plus the corresponding imports in routers/__init__.py, main.py,
  rag_service.py. Also removes the two empty-catch violations in
  semantic_cache.ts (no longer applicable).

The exact-key Convex `lib/response_cache/{internal_mutations,
internal_queries}.ts` cache stays — it is the actually-wired one and
is correctly org-scoped.
larryro added a commit that referenced this pull request May 8, 2026
…eout

Round-2 v15 confirmed: /config unauthenticated, /openapi.json + /docs +
/redoc unauthenticated, RAG container ran as root, default token baked
into image ENV, strict-mode env name diverged across the wire,
non-constant-time token compare, plus three SSRF-guard gaps.

services/rag/app/auth.py
- W7 #3: hmac.compare_digest replaces == on the bearer compare. Removes
  the dead-code EXEMPT_PATHS frozenset.

services/rag/app/routers/health.py
- W7 #1: split into public_router (`/`, `/health`) and protected_router
  (`/config`). main.py mounts the protected one under
  Depends(verify_internal_token). Old `router` re-export stays for
  backwards compat.

services/rag/app/main.py
- W7 #2: docs_url / redoc_url / openapi_url are None outside debug.
- W7 #4: CORS allow_credentials flipped to False (bearer rides
  Authorization, never cookies).
- W7 #1 wiring: mount health-public + health-protected separately.

services/rag/app/config.py
- W7 #8: require_custom_internal_token accepts BOTH
  RAG_REQUIRE_CUSTOM_INTERNAL_TOKEN and TALE_REQUIRE_CUSTOM_RAG_TOKEN
  via pydantic AliasChoices.

services/rag/Dockerfile + services/convex/Dockerfile
- W7 #5: RAG container runs as non-root (uid:gid 1001:1001 `app`).
  RAG ingests untrusted PDFs/DOCX through native parsers; biggest
  blast radius in the stack, now hardened.
- W7 #6: removed RAG_INTERNAL_TOKEN=tale-rag-dev-only ENV bake from
  both runtime + scratch-squash stages and the matching bake in
  services/convex/Dockerfile. Operators MUST supply via env / compose
  / k8s secret.

services/platform/convex/lib/helpers/rag_config.ts
- W7 #9 F1: `redirect: 'manual'` on every ragFetch.
- W7 #9 F2: added fc00::/7 (IPv6 ULA) to v6 blocklist (AWS IPv6 IMDSv2).
- W7 #9 F3: strip trailing `.` before hostname blocklist lookup.
- W7 #9 F4: re-validate URL per ragFetch invocation (DNS rebinding +
  env rotation mitigation).
- W7 #9 F9: deleted path.startsWith('http') override branch (future-
  bypass foot-gun).

services/platform/convex/agent_tools/rag/helpers/fetch_document_chunks.ts
- W7 #10: pass timeoutMs=60_000 (default 10s was a regression).
- Plus MAX_ITERATIONS=30 cap and "cursor did not advance" break to
  defend against an adversarial RAG response.
larryro added a commit that referenced this pull request May 8, 2026
…cies

Bundle of round-2-confirmed cross-tenant fixes plus the dead-code
delete of the semantic LLM response cache.

POLICY_TYPES drift (W6 #5)
- lib/shared/schemas/governance.ts now includes
  'data_classification_notice' to match the Convex enum, killing the
  `as const` cast at use-data-classification-notice.ts:50.

documents/compare_documents.ts (W6 #8)
- Convex `_storage` is a global namespace; org membership alone was
  not enough to gate `ctx.storage.getUrl`. Adds a JOIN through
  fileMetadata via the new internal query verifyStorageIdsBelongToOrg
  to confirm both `baseStorageId` and `comparisonStorageId` are owned
  by the caller's org. Refuses with a clear error otherwise. Pattern
  copied from agent_tools/documents/helpers/retrieve_document.ts.

file_metadata/actions.ts::checkFileRagStatuses (W6 #9)
- Was an unauthenticated public action that could flip any org's
  fileMetadata.ragStatus to `failed` via expireStaleRagQueue (DoS,
  pre-existing on `main`). Now requires `getAuthUser` and filters
  storageIds to ones owned by an org the caller is a member of via
  the new file_metadata.internal_queries.filterStorageIdsByCallerOrg.

governance/queries.ts (W6 #11)
- getPolicy + listPolicies now apply a member-readable allow-list
  (data_classification_notice, feature_flags, pii_config,
  chat_filter, personalization, upload_policy, default_models). All
  other types — login_policy.trustedProxies, password_policy,
  two_factor_policy, model_access.rules, budgets, retention_policy,
  moderation_provider.endpoint, system_prompt — are admin-only.
  listPolicies silently filters those out for non-admins.

semantic LLM response cache — DELETE (W6 #12 + #13)
- Round-2 v05 confirmed the lookup is structurally cross-tenant
  (filters only on agent_name, model, expires_at, similarity; ignores
  user_id / organization_id even though they're stored). The platform
  helpers `lookupSemanticCache` / `storeSemanticCacheAsync` had ZERO
  callers in the monorepo, the FastAPI router was mounted but
  unreachable from platform — a latent foot-gun primed for the next
  dev to wire up unaware. Deletes:

  - services/platform/convex/lib/response_cache/semantic_cache.ts
  - services/platform/convex/lib/response_cache/internal_actions.ts
  - services/rag/app/routers/llm_cache.py
  - services/rag/app/services/llm_response_cache.py

  Plus the corresponding imports in routers/__init__.py, main.py,
  rag_service.py. Also removes the two empty-catch violations in
  semantic_cache.ts (no longer applicable).

The exact-key Convex `lib/response_cache/{internal_mutations,
internal_queries}.ts` cache stays — it is the actually-wired one and
is correctly org-scoped.
larryro added a commit that referenced this pull request May 8, 2026
…eout

Round-2 v15 confirmed: /config unauthenticated, /openapi.json + /docs +
/redoc unauthenticated, RAG container ran as root, default token baked
into image ENV, strict-mode env name diverged across the wire,
non-constant-time token compare, plus three SSRF-guard gaps.

services/rag/app/auth.py
- W7 #3: hmac.compare_digest replaces == on the bearer compare. Removes
  the dead-code EXEMPT_PATHS frozenset.

services/rag/app/routers/health.py
- W7 #1: split into public_router (`/`, `/health`) and protected_router
  (`/config`). main.py mounts the protected one under
  Depends(verify_internal_token). Old `router` re-export stays for
  backwards compat.

services/rag/app/main.py
- W7 #2: docs_url / redoc_url / openapi_url are None outside debug.
- W7 #4: CORS allow_credentials flipped to False (bearer rides
  Authorization, never cookies).
- W7 #1 wiring: mount health-public + health-protected separately.

services/rag/app/config.py
- W7 #8: require_custom_internal_token accepts BOTH
  RAG_REQUIRE_CUSTOM_INTERNAL_TOKEN and TALE_REQUIRE_CUSTOM_RAG_TOKEN
  via pydantic AliasChoices.

services/rag/Dockerfile + services/convex/Dockerfile
- W7 #5: RAG container runs as non-root (uid:gid 1001:1001 `app`).
  RAG ingests untrusted PDFs/DOCX through native parsers; biggest
  blast radius in the stack, now hardened.
- W7 #6: removed RAG_INTERNAL_TOKEN=tale-rag-dev-only ENV bake from
  both runtime + scratch-squash stages and the matching bake in
  services/convex/Dockerfile. Operators MUST supply via env / compose
  / k8s secret.

services/platform/convex/lib/helpers/rag_config.ts
- W7 #9 F1: `redirect: 'manual'` on every ragFetch.
- W7 #9 F2: added fc00::/7 (IPv6 ULA) to v6 blocklist (AWS IPv6 IMDSv2).
- W7 #9 F3: strip trailing `.` before hostname blocklist lookup.
- W7 #9 F4: re-validate URL per ragFetch invocation (DNS rebinding +
  env rotation mitigation).
- W7 #9 F9: deleted path.startsWith('http') override branch (future-
  bypass foot-gun).

services/platform/convex/agent_tools/rag/helpers/fetch_document_chunks.ts
- W7 #10: pass timeoutMs=60_000 (default 10s was a regression).
- Plus MAX_ITERATIONS=30 cap and "cursor did not advance" break to
  defend against an adversarial RAG response.
larryro added a commit that referenced this pull request May 9, 2026
…inding

Five clustered cross-tenant + IDOR P0s flagged by the two-round
data-protection review (round-1 #5, #16, #17, #27, round-2 V1, V7, V8):

P0-1, P0-2: generic admin-trash restore
  `restoreRowToActive` (soft_delete_helpers) added required `organizationId`
  arg + `userMembershipIds` set; refuses cross-org rows with `not_found`
  (no foreign-id existence oracle) and rows whose author is on a custodian
  hold. Added `authorField` to `SOFT_DELETE_RESOURCE_CONFIG` per resource
  type so the cascade fires uniformly across the 14 restorable types.
  `restoreSoftDeletedRow` (governance/restore.ts) threads holds + orgId.

P0-3, P0-4: REST cross-tenant on GET/PATCH/DELETE
  Added optional `callerOrgId` arg to every internal query/mutation
  reachable from REST: threads (`getThreadMetadata`,
  `getThreadMessagesInternal`), customers (get/update/delete), vendors
  (get/update/delete), products (get/update/delete), documents
  (getDocumentByIdRaw, updateDocument, deleteDocumentById), workflow
  executions (getExecution, getRawExecution, listExecutionsCursorInternal,
  getExecutionStepJournalInternal), workflow triggers
  (cancelExecutionInternal, updateScheduleInternal, deleteScheduleInternal,
  deleteWebhookInternal). Each REST handler passes `rc.org.organizationId`.
  Mismatches return null/empty (so REST surfaces 404) — never throw
  cross-tenantly to avoid leaking row existence.

P0-5: rag_search IDOR + thread-binding
  Architectural change: chat uploads bind to their thread at upload time.
  - file_metadata schema: new optional `threadId` + composite index
    `by_organizationId_and_threadId`.
  - file_metadata.saveFileMetadata: accepts threadId; refuses cross-org
    threadId via threadMetadata lookup.
  - chat-input upload hook: passes the current chat threadId.
  - New helpers: `threads/get_thread_ancestor_chain.ts` (action-side walk
    of the delegation chain via existing `getParentThreadId`),
    `agent_tools/rag/helpers/verify_thread_scoped_access.ts` (query that
    same-org-checks each storageId, then if `meta.threadId` is set,
    requires it in the caller's accessible chain).
  - rag_search search-op: when explicit fileIds are supplied, verify
    via the new query (replaces unauthenticated short-circuit).
  - rag_search retrieve-op: replaces commit d7bc3da's stricter
    "agent allow-list" check, which over-strictly blocked legitimate
    chat-upload retrieval (chat uploads never appear in the agent's
    pre-configured `knowledgeFileIds`/`agentTeamId`/`includeOrgKnowledge`
    sets). Same-org + thread-scope is the correct invariant.

Cascade integration:
  - threads/cascade_helpers.ts: deletes bound fileMetadata + storage
    blobs as part of thread hard-delete (Pass B / GDPR erasure path).
    RAG-side fan-out is deferred to commit 4 (P0-13).
  - threads/delete_chat_thread.ts (Pass A trash): cascades soft-delete
    to bound fileMetadata rows (lifecycleStatus='trashed', mirrors
    grace-extension defense).
  - threads/restore_chat_thread.ts: cascades restore to fileMetadata
    rows trashed within ±5s skew of the thread's trash time (avoids
    zombie-restoring files that were independently trashed earlier).

Verified: `bun typecheck` clean; 871 tests across affected areas pass.
Regression tests for the new gates land in commit 6 (test guards) per
the bundle plan.
larryro added a commit that referenced this pull request May 9, 2026
…cies

Bundle of round-2-confirmed cross-tenant fixes plus the dead-code
delete of the semantic LLM response cache.

POLICY_TYPES drift (W6 #5)
- lib/shared/schemas/governance.ts now includes
  'data_classification_notice' to match the Convex enum, killing the
  `as const` cast at use-data-classification-notice.ts:50.

documents/compare_documents.ts (W6 #8)
- Convex `_storage` is a global namespace; org membership alone was
  not enough to gate `ctx.storage.getUrl`. Adds a JOIN through
  fileMetadata via the new internal query verifyStorageIdsBelongToOrg
  to confirm both `baseStorageId` and `comparisonStorageId` are owned
  by the caller's org. Refuses with a clear error otherwise. Pattern
  copied from agent_tools/documents/helpers/retrieve_document.ts.

file_metadata/actions.ts::checkFileRagStatuses (W6 #9)
- Was an unauthenticated public action that could flip any org's
  fileMetadata.ragStatus to `failed` via expireStaleRagQueue (DoS,
  pre-existing on `main`). Now requires `getAuthUser` and filters
  storageIds to ones owned by an org the caller is a member of via
  the new file_metadata.internal_queries.filterStorageIdsByCallerOrg.

governance/queries.ts (W6 #11)
- getPolicy + listPolicies now apply a member-readable allow-list
  (data_classification_notice, feature_flags, pii_config,
  chat_filter, personalization, upload_policy, default_models). All
  other types — login_policy.trustedProxies, password_policy,
  two_factor_policy, model_access.rules, budgets, retention_policy,
  moderation_provider.endpoint, system_prompt — are admin-only.
  listPolicies silently filters those out for non-admins.

semantic LLM response cache — DELETE (W6 #12 + #13)
- Round-2 v05 confirmed the lookup is structurally cross-tenant
  (filters only on agent_name, model, expires_at, similarity; ignores
  user_id / organization_id even though they're stored). The platform
  helpers `lookupSemanticCache` / `storeSemanticCacheAsync` had ZERO
  callers in the monorepo, the FastAPI router was mounted but
  unreachable from platform — a latent foot-gun primed for the next
  dev to wire up unaware. Deletes:

  - services/platform/convex/lib/response_cache/semantic_cache.ts
  - services/platform/convex/lib/response_cache/internal_actions.ts
  - services/rag/app/routers/llm_cache.py
  - services/rag/app/services/llm_response_cache.py

  Plus the corresponding imports in routers/__init__.py, main.py,
  rag_service.py. Also removes the two empty-catch violations in
  semantic_cache.ts (no longer applicable).

The exact-key Convex `lib/response_cache/{internal_mutations,
internal_queries}.ts` cache stays — it is the actually-wired one and
is correctly org-scoped.
larryro added a commit that referenced this pull request May 9, 2026
…eout

Round-2 v15 confirmed: /config unauthenticated, /openapi.json + /docs +
/redoc unauthenticated, RAG container ran as root, default token baked
into image ENV, strict-mode env name diverged across the wire,
non-constant-time token compare, plus three SSRF-guard gaps.

services/rag/app/auth.py
- W7 #3: hmac.compare_digest replaces == on the bearer compare. Removes
  the dead-code EXEMPT_PATHS frozenset.

services/rag/app/routers/health.py
- W7 #1: split into public_router (`/`, `/health`) and protected_router
  (`/config`). main.py mounts the protected one under
  Depends(verify_internal_token). Old `router` re-export stays for
  backwards compat.

services/rag/app/main.py
- W7 #2: docs_url / redoc_url / openapi_url are None outside debug.
- W7 #4: CORS allow_credentials flipped to False (bearer rides
  Authorization, never cookies).
- W7 #1 wiring: mount health-public + health-protected separately.

services/rag/app/config.py
- W7 #8: require_custom_internal_token accepts BOTH
  RAG_REQUIRE_CUSTOM_INTERNAL_TOKEN and TALE_REQUIRE_CUSTOM_RAG_TOKEN
  via pydantic AliasChoices.

services/rag/Dockerfile + services/convex/Dockerfile
- W7 #5: RAG container runs as non-root (uid:gid 1001:1001 `app`).
  RAG ingests untrusted PDFs/DOCX through native parsers; biggest
  blast radius in the stack, now hardened.
- W7 #6: removed RAG_INTERNAL_TOKEN=tale-rag-dev-only ENV bake from
  both runtime + scratch-squash stages and the matching bake in
  services/convex/Dockerfile. Operators MUST supply via env / compose
  / k8s secret.

services/platform/convex/lib/helpers/rag_config.ts
- W7 #9 F1: `redirect: 'manual'` on every ragFetch.
- W7 #9 F2: added fc00::/7 (IPv6 ULA) to v6 blocklist (AWS IPv6 IMDSv2).
- W7 #9 F3: strip trailing `.` before hostname blocklist lookup.
- W7 #9 F4: re-validate URL per ragFetch invocation (DNS rebinding +
  env rotation mitigation).
- W7 #9 F9: deleted path.startsWith('http') override branch (future-
  bypass foot-gun).

services/platform/convex/agent_tools/rag/helpers/fetch_document_chunks.ts
- W7 #10: pass timeoutMs=60_000 (default 10s was a regression).
- Plus MAX_ITERATIONS=30 cap and "cursor did not advance" break to
  defend against an adversarial RAG response.
larryro added a commit that referenced this pull request May 9, 2026
…inding

Five clustered cross-tenant + IDOR P0s flagged by the two-round
data-protection review (round-1 #5, #16, #17, #27, round-2 V1, V7, V8):

P0-1, P0-2: generic admin-trash restore
  `restoreRowToActive` (soft_delete_helpers) added required `organizationId`
  arg + `userMembershipIds` set; refuses cross-org rows with `not_found`
  (no foreign-id existence oracle) and rows whose author is on a custodian
  hold. Added `authorField` to `SOFT_DELETE_RESOURCE_CONFIG` per resource
  type so the cascade fires uniformly across the 14 restorable types.
  `restoreSoftDeletedRow` (governance/restore.ts) threads holds + orgId.

P0-3, P0-4: REST cross-tenant on GET/PATCH/DELETE
  Added optional `callerOrgId` arg to every internal query/mutation
  reachable from REST: threads (`getThreadMetadata`,
  `getThreadMessagesInternal`), customers (get/update/delete), vendors
  (get/update/delete), products (get/update/delete), documents
  (getDocumentByIdRaw, updateDocument, deleteDocumentById), workflow
  executions (getExecution, getRawExecution, listExecutionsCursorInternal,
  getExecutionStepJournalInternal), workflow triggers
  (cancelExecutionInternal, updateScheduleInternal, deleteScheduleInternal,
  deleteWebhookInternal). Each REST handler passes `rc.org.organizationId`.
  Mismatches return null/empty (so REST surfaces 404) — never throw
  cross-tenantly to avoid leaking row existence.

P0-5: rag_search IDOR + thread-binding
  Architectural change: chat uploads bind to their thread at upload time.
  - file_metadata schema: new optional `threadId` + composite index
    `by_organizationId_and_threadId`.
  - file_metadata.saveFileMetadata: accepts threadId; refuses cross-org
    threadId via threadMetadata lookup.
  - chat-input upload hook: passes the current chat threadId.
  - New helpers: `threads/get_thread_ancestor_chain.ts` (action-side walk
    of the delegation chain via existing `getParentThreadId`),
    `agent_tools/rag/helpers/verify_thread_scoped_access.ts` (query that
    same-org-checks each storageId, then if `meta.threadId` is set,
    requires it in the caller's accessible chain).
  - rag_search search-op: when explicit fileIds are supplied, verify
    via the new query (replaces unauthenticated short-circuit).
  - rag_search retrieve-op: replaces commit d7bc3da's stricter
    "agent allow-list" check, which over-strictly blocked legitimate
    chat-upload retrieval (chat uploads never appear in the agent's
    pre-configured `knowledgeFileIds`/`agentTeamId`/`includeOrgKnowledge`
    sets). Same-org + thread-scope is the correct invariant.

Cascade integration:
  - threads/cascade_helpers.ts: deletes bound fileMetadata + storage
    blobs as part of thread hard-delete (Pass B / GDPR erasure path).
    RAG-side fan-out is deferred to commit 4 (P0-13).
  - threads/delete_chat_thread.ts (Pass A trash): cascades soft-delete
    to bound fileMetadata rows (lifecycleStatus='trashed', mirrors
    grace-extension defense).
  - threads/restore_chat_thread.ts: cascades restore to fileMetadata
    rows trashed within ±5s skew of the thread's trash time (avoids
    zombie-restoring files that were independently trashed earlier).

Verified: `bun typecheck` clean; 871 tests across affected areas pass.
Regression tests for the new gates land in commit 6 (test guards) per
the bundle plan.
larryro added a commit that referenced this pull request May 17, 2026
Closes #5, #6, #7, #8, #40 — backend billing / ledger correctness.

- `checkRuleAgainstUsage` and `collectWarnings` gain a symmetric
  `prospectiveRequests` parameter. The post-ledger TTS call site
  (`reserveChunk` → `checkBudget`) passes `1` so an admin who set
  `maxRequests` for the period sees parallel chunks of a single
  message honour the cap the same way `maxCostCents` already did.
  LLM call sites default to 0 — their ledger write is synchronous
  so retrospective checks stay accurate.
- `reserveChunk` overwrite branch: `agentSlug` now falls back to
  `existing.agentSlug` when the thread temporarily reports no agent
  on a retry (agent detached between attempts). Without the fallback,
  ledger writes for the retry landed under the TTS_SLUG sentinel and
  Top Agents analytics drifted across retries.
- `markChunkReadyAndRecordUsage` moves the `!row.userId` check ABOVE
  the `status: 'ready'` patch. A pending row missing userId now flips
  to `'failed'` with a `PROVIDER_ERROR` code instead of silently
  becoming playable while skipping the ledger write. The
  `synthesize.ts` compensating block handles the resulting throw,
  deleting the just-uploaded blob and reporting the failure to the
  client.
- `markChunkFailed` now schedules `maybeCleanupChunks` on `index === 0`
  too, matching the success path. A message whose chunk 0 always fails
  used to leave the daily cron as the only backstop; the `cleanup:tts`
  limiter still gates the schedule so a burst of failures can't flood
  the dispatcher.
- `recordTtsUsageInline` adds an in-memory `provider` filter on the
  upsert lookup. A TTS row with `provider: 'openai'` no longer merges
  into a sibling LLM row that happens to share (org, user, period,
  team, agent, model) under a different provider. Latent today on
  single-TTS-provider configs; load-bearing once a second TTS
  provider ships. A structural fix (extend the index to include
  `provider`) is tracked as a follow-up.
larryro added a commit that referenced this pull request May 17, 2026
Closes #5, #6, #7, #8, #40 — backend billing / ledger correctness.

- `checkRuleAgainstUsage` and `collectWarnings` gain a symmetric
  `prospectiveRequests` parameter. The post-ledger TTS call site
  (`reserveChunk` → `checkBudget`) passes `1` so an admin who set
  `maxRequests` for the period sees parallel chunks of a single
  message honour the cap the same way `maxCostCents` already did.
  LLM call sites default to 0 — their ledger write is synchronous
  so retrospective checks stay accurate.
- `reserveChunk` overwrite branch: `agentSlug` now falls back to
  `existing.agentSlug` when the thread temporarily reports no agent
  on a retry (agent detached between attempts). Without the fallback,
  ledger writes for the retry landed under the TTS_SLUG sentinel and
  Top Agents analytics drifted across retries.
- `markChunkReadyAndRecordUsage` moves the `!row.userId` check ABOVE
  the `status: 'ready'` patch. A pending row missing userId now flips
  to `'failed'` with a `PROVIDER_ERROR` code instead of silently
  becoming playable while skipping the ledger write. The
  `synthesize.ts` compensating block handles the resulting throw,
  deleting the just-uploaded blob and reporting the failure to the
  client.
- `markChunkFailed` now schedules `maybeCleanupChunks` on `index === 0`
  too, matching the success path. A message whose chunk 0 always fails
  used to leave the daily cron as the only backstop; the `cleanup:tts`
  limiter still gates the schedule so a burst of failures can't flood
  the dispatcher.
- `recordTtsUsageInline` adds an in-memory `provider` filter on the
  upsert lookup. A TTS row with `provider: 'openai'` no longer merges
  into a sibling LLM row that happens to share (org, user, period,
  team, agent, model) under a different provider. Latent today on
  single-TTS-provider configs; load-bearing once a second TTS
  provider ships. A structural fix (extend the index to include
  `provider`) is tracked as a follow-up.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant