feat(search): add list fields endpoint + pagination#3
Draft
feat(search): add list fields endpoint + pagination#3
Conversation
|
New Issues (7)Checkmarx found the following issues in this Pull Request
Fixed Issues (5)Great job! The following issues were fixed in this Pull Request
|
Totodore
force-pushed
the
feat-list-fields-quickwit
branch
3 times, most recently
from
5e03d9b to
79c11e2
Compare
Totodore
force-pushed
the
feat-list-fields-quickwit
branch
from
79c11e2 to
1288bfb
Compare
|
YARN is no longer allowed. Kindly replace the lockfile using PNPM. Found in ./quickwit/quickwit-ui/yarn.lock |
Rustfmt 1.9.0 (shipped in nightly 2026-03-01) changed behavior for `imports_granularity = "Module"`: top-level imports are no longer merged. Expand grouped `pub use` re-exports and fix sort order. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* Remove unused React imports from test mocks The automatic JSX transform doesn't require importing React. These unused imports cause biome lint errors. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update quickwit-ui dependencies Bump minor/patch versions: - @babel/core 7.28.5 → 7.29.0 - @biomejs/biome 2.3.5 → 2.4.4 - @mui/* 7.3.5 → 7.3.8, x-charts/x-date-pickers 8.18.0 → 8.27.x - react/react-dom 19.2.0 → 19.2.4 - react-router 7.12.0 → 7.13.1 - vite 7.2.2 → 7.3.1 - monaco-editor 0.54.0 → 0.55.1 - dayjs 1.11.7 → 1.11.19 - swagger-ui-react 5.30.2 → 5.32.0 - dompurify (resolution) 3.2.4 → 3.3.1 - and other minor bumps Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix stale apt cache in UI CI Cypress step Add apt-get update before installing protobuf-compiler to avoid 404 errors from stale package mirrors on GitHub Actions runners. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Remove unused dependencies react-monaco-editor and web-vitals react-monaco-editor was never imported (project uses @monaco-editor/react). web-vitals and reportWebVitals.ts were dead CRA boilerplate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Migrate e2e tests from Cypress to Playwright - Replace Cypress with @playwright/test - Create playwright.config.ts with Chromium + baseURL - Translate 3 existing tests + add search page test - Use --only-shell for lighter Chromium install in CI - Exclude e2e/ from Jest via testPathIgnorePatterns Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Fix biome formatting in homepage.spec.ts Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Refactor UI CI: replace matrix with dedicated jobs Split the matrix-based workflow into two explicit jobs: - `checks`: lint, type check, unit tests (Node-only, no Rust/Postgres) - `e2e`: Playwright tests (with Rust build, Postgres, Chromium) This avoids spinning up Postgres and installing Rust for the lint/type/test jobs that don't need them. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Share Rust cache between CI and UI CI workflows Align RUSTFLAGS in ui-ci.yml with ci.yml and add a shared-key to Swatinem/rust-cache so both workflows reuse the same cached artifacts. This avoids a cold Rust build in the UI e2e job when the main CI has already built on the same branch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* make lambda support optional * map lambda error to actual variant * retry rate limited lambdas
…#6168) * WIP * Add `--debug` option to REST API tests runner * Claude WIP * Support Trino ES connector's bool query and range query extensions - Accept `adjust_pure_negative` field in bool queries (emitted by ES Java BoolQueryBuilder, blocks all WHERE predicate pushdown without this fix) - Accept legacy `from`/`to`/`include_lower`/`include_upper` fields in range queries (used by ES Java RangeQueryBuilder instead of standard gt/gte/lt/lte) Co-authored-by: Cursor <cursoragent@cursor.com> * Support wildcard and multi-index patterns in _mappings endpoint The _mapping(s) endpoint now resolves wildcard (`*`) and comma-separated index patterns against the metastore, matching the behavior of real Elasticsearch. This enables Trino's wildcard table feature (e.g. `SELECT * FROM "stack*"`), which calls `GET /stack*/_mappings` to discover schemas across matching indices. Co-authored-by: Cursor <cursoragent@cursor.com> * Extract inline warp filters into filter.rs and revert debug additions Move the inline warp path filters for _aliases and _mappings handlers into dedicated functions in filter.rs, consistent with all other ES compat endpoints. Revert the --debug CLI flag added to run_tests.py as per-step debug: true in YAML files is sufficient. Co-authored-by: Cursor <cursoragent@cursor.com> * Fix clippy collapsible_if and rustfmt formatting Collapse nested if-let + if into if-let chains in range query conversion logic. Fix line length and import ordering for rustfmt. Co-authored-by: Cursor <cursoragent@cursor.com> * Address PR quickwit-oss#6168 review comments - bool_query: rename adjust_pure_negative to _adjust_pure_negative with IgnoredAny to convey the field is accepted but unused - bool_query: remove unused let binding in test - range_query: rename json_number to into_json_number, remove doc comments on legacy fields, uncomment from/to conversion logic - cat_indices: add unit test for s parameter validation - mappings: add comprehensive tests for build_properties and merge_dynamic_fields - rest_handler: use advertise_addr for publish_address in _nodes/http and fix es_compat_delete_scroll_handler formatting Made-with: Cursor * bool_query: derive PartialEq instead of manual impl IgnoredAny implements PartialEq (but not Eq), so PartialEq can be derived. Only Eq needs a manual impl since IgnoredAny is a unit struct. Made-with: Cursor * Fix rustfmt nightly formatting for es_compat_delete_scroll_handler Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Adrien Guillo <adrien.guillo@datadoghq.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…ickwit-oss#6132 (quickwit-oss#6191) Co-authored-by: Nicolas Coiffier <n.coiffier@celonis.com>
* feat(obs): export obs as otel * fix: typo * fix: dd license tool
Navigate directly to `/ui` instead of relying on the root redirect, making the tests resilient to changes in the root redirect target. Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
…uickwit-oss#6102) * feat(es-compat): add index_filter support for field capabilities API Implements index_filter parameter support for the ES-compatible _field_caps endpoint, allowing users to filter field capabilities based on document queries. Changes: - Add query_ast field to ListFieldsRequest and LeafListFieldsRequest protos - Parse index_filter from ES Query DSL and convert to QueryAst - Pass query_ast through to leaf nodes for future filtering support - Add unit tests for index_filter parsing - Add REST API integration tests Note: This implementation accepts and parses the index_filter parameter for API compatibility. Full split-level document filtering will be added as a follow-up enhancement. Closes quickwit-oss#5693 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: ruo <ruoliu.dev@gmail.com> * feat(es-compat): implement split-level filtering for field_caps index_filter Address PR review comments for index_filter support in _field_caps API: - Extract `parse_index_filter_to_query_ast()` function with clean prototype - Implement split-level filtering via `split_matches_query()` using lightweight `query.count()` execution (no document materialization) - Add proper async handling with ByteRangeCache, warmup(), and run_cpu_intensive() for Quickwit's async-only storage - Add metastore-level pruning: - Tag extraction via `extract_tags_from_query()` - Time range extraction via `refine_start_end_timestamp_from_ast()` - Build DocMapper only when query_ast is provided (no overhead for common path without index_filter) - Fix REST API tests: use `json:` key (not `json_body:`), use lowercase term values to match tokenizer behavior - Update tests to run against both quickwit and elasticsearch engines Two-level filtering now implemented: 1. Metastore level: tags + time range from query AST 2. Split level: lightweight query execution for accurate filtering Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: ruo <ruoliu.dev@gmail.com> * refactor(es-compat): use best-effort metadata filtering for index_filter Remove heavy split-level query execution for field_caps index_filter. The implementation now aligns with ES's "best-effort" approach that uses metadata-level filtering only (time range, tags) instead of opening splits and executing queries. Changes: - Remove split_matches_query function (no longer opens splits) - Remove query_ast and doc_mapper from LeafListFieldsRequest proto - Keep metadata-level filtering in root_list_fields: - Time range extraction from query AST - Tag-based split pruning - Simplify leaf_list_fields to just return fields from all splits This matches ES semantics: "filtering is done on a best-effort basis... this API may return an index even if the provided filter matches no document." Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: ruo <ruoliu.dev@gmail.com> * fix(es-compat): reject empty index_filter to match ES behavior - Remove empty object {} handling in parse_index_filter_to_query_ast - ES rejects empty index_filter with 400, now QW does too - Add tag_fields config and tag-based index_filter test - Update unit and integration tests accordingly Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> Signed-off-by: ruo <ruoliu.dev@gmail.com> * Added ref doc for the new functionality. * cargo fmt fix --------- Signed-off-by: ruo <ruoliu.dev@gmail.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: fulmicoton <paul.masurel@datadoghq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Describe the proposed changes made in this PR.
How was this PR tested?
Describe how you tested this PR.