feat: add sorted_series column for DataFusion streaming aggregation

[g-talbot]

Summary

• Compute a composite, lexicographically sortable sorted_series binary column at Parquet write time using storekey order-preserving encoding
• For each row, encodes non-null sort schema tag columns as (ordinal: u8, value: str) pairs, then appends (ordinal: u8, timeseries_id: i64) as final discriminator
• Identical timeseries always produce identical byte keys regardless of timestamp or value, enabling DataFusion's streaming AggregateExec and BoundedWindowAggExec with O(1) memory instead of O(N) hash tables
• Column is placed after sort columns in physical layout (Phase 1b in reorder_columns) for optimal streaming read
• Fixes create_nullable_dict_array bug: dictionary keys now correctly index into unique values (was using original array index, causing panics for mixed null/non-null inputs)
• timeseries_id is mandatory — batches without it are rejected as malformed
• ParquetWriter::new returns Result — invalid sort fields are a hard error, not a silent degradation
• Writer sorts with nulls_first=false so sorted_series keys are monotonic with physical sort order

Stacked on top of #6287 (column ordering) and timeseries_id work.

Design

Based on the Sorted Series Column design doc:

Key structure for sort schema [metric_name(0), service(1), ..., host(5), timeseries_id(6)]:

┌──────────┬────────────────┬──────────┬──────────────┬──────────┬─────────────────┐
│ ordinal 0│ "cpu.usage"    │ ordinal 1│ "api"        │ ordinal 6│ timeseries_id   │
│ (u8)     │ (storekey str) │ (u8)     │ (storekey)   │ (u8)     │ (storekey i64)  │
└──────────┴────────────────┴──────────┴──────────────┴──────────┴─────────────────┘

Null tag columns are skipped (no ordinal or value emitted). The ordinal prefix prevents cross-column byte collisions for sparse schemas. timeseries_id is always present as the final discriminator — it is the only guaranteed column that distinguishes series with identical tags.

Nulls sort last (nulls_first=false) in the physical sort order, which matches the key encoding: a skipped (null) ordinal produces a shorter key that compares before a present ordinal, and nulls-last ensures the physical row order agrees.

Test plan

• 216 tests pass on quickwit-parquet-engine (identity, discrimination, sort-order, null handling, stability, Parquet round-trip, structural ordinal verification, monotonicity, mandatory timeseries_id, proptests)
• 3 E2E pipeline tests pass on quickwit-indexing (metrics, sketch, file-backed metastore)
• Clippy clean (zero warnings), formatted, no unused deps
• License headers pass
• Docs compile

🤖 Generated with Claude Code

[mattmkim]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 652d128d5e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mattmkim]

commit history is messed up, pushed existing history to https://github.com/quickwit-oss/quickwit/tree/matthew.kim/gtt/sorted-series-key just in case, going to cherry pick the commits we want

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 069a5a1871

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".