Skip to content

feat: expose distinct warehouse event names per connection - #8231

Merged
Zaimwa9 merged 9 commits into
mainfrom
feat/get-distinct-events-from-warehouse
Aug 6, 2026
Merged

feat: expose distinct warehouse event names per connection#8231
Zaimwa9 merged 9 commits into
mainfrom
feat/get-distinct-events-from-warehouse

Conversation

@Zaimwa9

@Zaimwa9 Zaimwa9 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor
  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Adds GET /environments/{key}/warehouse-connections/{id}/events/, returning the distinct event names recorded in the connection's warehouse so metric creation can offer them for selection.

  • Dispatches on warehouse type: managed ClickHouse queried directly, BYO ClickHouse via delivery_client with a 5-minute per-connection cache (successes and failures).
  • Most recently seen first, capped at 500, is_truncated flag as the hook for future pagination.
  • Pagination deferred: brand-new feature, and a single cached query contains warehouse costs.
  • 503 when unreachable, 400 for unsupported types; read throttle scope applies.

How did you test this code?

Parametrised unit tests (dispatch, truncation, caching, response mapping); manual check against a ClickHouse Cloud instance via a BYO connection.

@Zaimwa9
Zaimwa9 requested review from a team as code owners August 6, 2026 08:57
@Zaimwa9
Zaimwa9 requested review from emyller and removed request for a team August 6, 2026 08:57
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
docs Ready Ready Preview Aug 6, 2026 12:16pm
2 Skipped Deployments
Project Deployment Actions Updated (UTC)
flagsmith-frontend-preview Ignored Ignored Preview Aug 6, 2026 12:16pm
flagsmith-frontend-staging Ignored Ignored Preview Aug 6, 2026 12:16pm

Request Review

@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Added WarehouseEventNames and bounded, recent-first warehouse event retrieval with truncation metadata. Added separate success and failure caching for customer ClickHouse retrieval and failure logging. Added a throttled events endpoint with validation and HTTP 503 handling. Added service and view tests. Updated warehouse observability event catalogue references.

Estimated code review effort: 3 (Moderate) | ~20 minutes


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added api Issue related to the REST API docs Documentation updates labels Aug 6, 2026
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Docker builds report

Image Build Status Security report
ghcr.io/flagsmith/flagsmith-e2e:pr-8231 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-api-test:pr-8231 Finished ✅ Skipped
ghcr.io/flagsmith/flagsmith-frontend:pr-8231 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-api:pr-8231 Finished ✅ Results
ghcr.io/flagsmith/flagsmith:pr-8231 Finished ✅ Results
ghcr.io/flagsmith/flagsmith-private-cloud:pr-8231 Finished ✅ Results

@github-actions github-actions Bot added feature New feature or request and removed docs Documentation updates labels Aug 6, 2026
@flagsmith-engineering
flagsmith-engineering Bot requested a review from a team as a code owner August 6, 2026 08:58
@github-actions github-actions Bot added the docs Documentation updates label Aug 6, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c52d9b3b-e0b8-4d2e-b9e1-d6a267aaad7e

📥 Commits

Reviewing files that changed from the base of the PR and between 377f203 and f440ace.

📒 Files selected for processing (6)
  • api/experimentation/dataclasses.py
  • api/experimentation/services.py
  • api/experimentation/views.py
  • api/tests/unit/experimentation/test_services.py
  • api/tests/unit/experimentation/test_views.py
  • docs/docs/deployment-self-hosting/observability/_events-catalogue.md

Comment thread api/experimentation/services.py Outdated
Comment thread docs/docs/deployment-self-hosting/observability/_events-catalogue.md Outdated
@Zaimwa9
Zaimwa9 removed the request for review from emyller August 6, 2026 09:01

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: c0677e53-25d8-4fc2-8cab-b5f948dcfe83

📥 Commits

Reviewing files that changed from the base of the PR and between f440ace and 76d3aa2.

📒 Files selected for processing (2)
  • docs/docs/deployment-self-hosting/observability/_events-catalogue.md
  • openapi.yaml

Comment thread openapi.yaml
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor
✅ private-cloud · depot-ubuntu-latest-16 — run #19077 (attempt 2)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  4 passed

Details

stats  4 tests across 4 suites
duration  32.5 seconds
commit  bffe653
info  🔄 Run: #19077 (attempt 2)

🗂️ Previous results
❌ private-cloud · depot-ubuntu-latest-16 — run #19077 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

failed  1 failed
passed  1 passed

Details

stats  2 tests across 2 suites
duration  1 minute, 20 seconds
commit  bffe653
info  📦 Artifacts: View test results and HTML report
🔄 Run: #19077 (attempt 1)

Failed tests

firefox › tests/environment-permission-test.pw.ts › Environment Permission Tests › Environment-level permissions control access to features, identities, and segments @enterprise

✅ private-cloud · depot-ubuntu-latest-arm-16 — run #19077 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  4 passed

Details

stats  4 tests across 4 suites
duration  12.1 seconds
commit  bffe653
info  🔄 Run: #19077 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #19077 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  44.6 seconds
commit  bffe653
info  🔄 Run: #19077 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #19077 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  40.9 seconds
commit  bffe653
info  🔄 Run: #19077 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-16 — run #19054 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  56.2 seconds
commit  56ca574
info  🔄 Run: #19054 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-arm-16 — run #19054 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  36.1 seconds
commit  56ca574
info  🔄 Run: #19054 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #19054 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  32.7 seconds
commit  56ca574
info  🔄 Run: #19054 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #19055 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  44.2 seconds
commit  d59503b
info  🔄 Run: #19055 (attempt 1)

✅ oss · depot-ubuntu-latest-arm-16 — run #19054 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-arm-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  37.4 seconds
commit  56ca574
info  🔄 Run: #19054 (attempt 1)

✅ oss · depot-ubuntu-latest-16 — run #19055 (attempt 1)

Playwright Test Results (oss - depot-ubuntu-latest-16)

passed  1 passed

Details

stats  1 test across 1 suite
duration  41 seconds
commit  d59503b
info  🔄 Run: #19055 (attempt 1)

✅ private-cloud · depot-ubuntu-latest-arm-16 — run #19052 (attempt 1)

Playwright Test Results (private-cloud - depot-ubuntu-latest-arm-16)

passed  2 passed

Details

stats  2 tests across 2 suites
duration  36.8 seconds
commit  57fa33d
info  🔄 Run: #19052 (attempt 1)

@codecov

codecov Bot commented Aug 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.72%. Comparing base (377f203) to head (bffe653).

Additional details and impacted files
@@           Coverage Diff            @@
##             main    #8231    +/-   ##
========================================
  Coverage   98.72%   98.72%            
========================================
  Files        1543     1543            
  Lines       61683    61805   +122     
========================================
+ Hits        60895    61017   +122     
  Misses        788      788            

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Visual Regression

19 screenshots compared. See report for details.
View full report

@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Aug 6, 2026
@github-actions github-actions Bot added docs Documentation updates and removed feature New feature or request labels Aug 6, 2026
@github-actions github-actions Bot added feature New feature or request and removed docs Documentation updates labels Aug 6, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
api/experimentation/views.py (1)

231-241: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Declare the 400 and 503 responses for warehouse_connections/{connection_id}/events.

The endpoint can return {"detail": "Event listing is not supported for this warehouse type."} when warehouse_type is unsupported, and {"detail": "The warehouse is currently unreachable."} when event names are unavailable. Update @extend_schema with 400 and 503 responses and regenerate openapi.yaml.


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 22173458-6db0-4456-b53c-6192d3537e31

📥 Commits

Reviewing files that changed from the base of the PR and between 3fe89ea and 57fa33d.

📒 Files selected for processing (3)
  • api/experimentation/services.py
  • api/experimentation/views.py
  • api/tests/unit/experimentation/test_views.py

Comment thread api/experimentation/services.py Outdated
Comment thread api/experimentation/views.py Outdated
Comment thread api/tests/unit/experimentation/test_views.py Outdated
@Zaimwa9

Zaimwa9 commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

@themis-blindfold review

@themis-blindfold

Copy link
Copy Markdown
Contributor

⚖️ Themis review: ✅ Ship it

Solid addition: a new read endpoint for warehouse event names, properly gated behind the existing feature flag and permission class, with per-connection caching for BYO ClickHouse and cache invalidation wired into the update path. The dispatch logic (managed vs. customer ClickHouse) mirrors the existing event stats pattern, the truncation detection via LIMIT+1 is clean, and the test matrix covers both warehouse types, failure caching, truncation, and cache clearing on credential changes.

Area Score
🎯 Correctness 5/5
🧪 Test coverage 4/5
📐 Code quality 5/5
🚀 Product impact 4/5
📝 Walkthrough
  • Event names service layer — new get_warehouse_event_names dispatches to managed ClickHouse (direct query) or customer ClickHouse (cached via delivery_client); shared query, params, and builder extracted for reuse across both paths.
  • Cache lifecycleclear_customer_warehouse_caches drops both event-stats and event-names cache entries; called from perform_update whenever config or credentials change, before the ClickHouse verification step.
  • View actionevents detail action on WarehouseConnectionViewSet; returns 400 for unsupported types, 503 when unreachable, 200 with {events, is_truncated} otherwise.
  • Tests — parametrised service tests for managed/BYO paths, truncation at 500, failure caching, and a view-level test confirming cache keys are cleared on PATCH.
  • Docs & OpenAPI — events catalogue updated with the new connection.event_names_failed event and adjusted line numbers; openapi.yaml gains the new path and WarehouseEventNamesResult schema.
🧪 How to verify
  1. Run pytest api/tests/unit/experimentation/test_services.py api/tests/unit/experimentation/test_views.py -x -q — all new and existing warehouse tests pass.
  2. Against a BYO ClickHouse connection, call GET /api/v1/environments/{key}/warehouse-connections/{id}/events/ twice within 5 minutes — second call should be a cache hit (no new connection opened on the customer host).
  3. PATCH the same connection's credentials, then GET events again — verify the response reflects the new warehouse, not a stale cache.
  4. Create a Snowflake-type connection and call the events endpoint — expect 400.
  5. Stop the BYO ClickHouse instance and call events — expect 503; repeat within 60 s and confirm the failure is cached (no new connection attempt in logs).

Automate: steps 1–5 are covered by the existing parametrised unit tests; a staging smoke test against a real BYO ClickHouse would add confidence.

Product take: This unlocks event-name selection in the metric creation UI, replacing a manual entry step with a dropdown sourced from the warehouse. Solid improvement for the experimentation workflow — not a capability expansion, but a meaningful UX upgrade that reduces setup friction and typo-driven misconfiguration.

🧭 Assumptions & unverified claims
  • The GROUP BY event ORDER BY max(timestamp) DESC query performs acceptably within the 5-second send_receive_timeout on customer ClickHouse instances with high event cardinality. The timeout bounds the blast radius, but a slow query would surface as a 503 rather than a partial result.
  • Django's default cache backend preserves dataclass identity through serialisation (pickle); a JSON-based backend would cause cache misses on every request for the BYO path. This is the same assumption the existing event-stats caching makes.

A cache that knows when to forget is worth more than one that never does · reviewed at 2b3617c

@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Aug 6, 2026
@github-actions github-actions Bot added the docs Documentation updates label Aug 6, 2026
Comment thread api/experimentation/services.py
Comment thread api/experimentation/services.py
Comment thread api/experimentation/services.py
Comment thread api/experimentation/services.py Outdated
Comment thread api/experimentation/services.py
Comment thread api/experimentation/services.py
Comment thread api/experimentation/views.py Outdated
@github-actions github-actions Bot added feature New feature or request and removed feature New feature or request docs Documentation updates labels Aug 6, 2026
@Zaimwa9
Zaimwa9 merged commit e9de7ba into main Aug 6, 2026
48 of 49 checks passed
@Zaimwa9
Zaimwa9 deleted the feat/get-distinct-events-from-warehouse branch August 6, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api Issue related to the REST API feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants