Skip to content

fix(planner): carry projections and early_limit into the physical plan - #207

Merged
hyperpolymath merged 2 commits into
mainfrom
fix/planner-carry-projections
Jul 28, 2026
Merged

fix(planner): carry projections and early_limit into the physical plan#207
hyperpolymath merged 2 commits into
mainfrom
fix/planner-carry-projections

Conversation

@hyperpolymath

Copy link
Copy Markdown
Owner

fix(planner): carry projections and early_limit into the physical plan

PlanNode has four fields; PlanStep had six, and neither projections nor
early_limit was among them. The optimizer computed both, then dropped them at
the logical -> physical boundary, so the executor never saw either pushdown.

That is worse than merely lossy, and this is the part worth flagging.
CostModel::estimate already discounts a node carrying an early limit
(cost.rs:345-350):

if let Some(limit) = node.early_limit {
    let limit_factor = (limit as f64 / 1000.0).min(1.0);
    selectivity *= limit_factor;
    time_ms *= 0.5 + 0.5 * limit_factor; // At least 50% of base cost
}

So a plan was priced as cheap as half its base cost because of a limit
pushdown that the physical plan had no way to perform. The planner was
rewarding an optimisation it then discarded. Nothing failed loudly — plans
simply cost less on paper than they cost to run, which is the kind of defect
that shows up as unexplained latency rather than as an error.

Measured before the change: early_limit had exactly one non-test read in the
entire tree — the cost adjustment above. projections had none; nothing
read it after vcl_bridge.rs built it.

Both fields are now on PlanStep and populated from the node. They carry
#[serde(default)], so a PhysicalPlan serialised before this change still
deserialises (absent -> vec![] / None, matching the old behaviour rather
than inventing a limit).

This surfaced while discharging optimize_is_permutation (#206). The Coq model
abstracts a node as (modality, list condition) — precisely the projection that
omits the two fields being lost. The abstraction was not wrong, but it was drawn
around the bug, which is why proving the permutation property could not have
caught it.

Test: test_pushdown_survives_optimization asserts both fields survive, and
matches steps by modality rather than index — asserting on steps[0] would test
the sort order, not the pushdown. Verified as a real guard, not a vacuous one:
with the population reverted it fails with projection pushdown was dropped at the physical boundary; with the fix it passes.

Verified: cargo test -p verisim-planner 118 passed / 0 failed; cargo clippy -p verisim-planner --all-targets clean; cargo check -p verisim-api --all-targets clean (it maps PlanStep into its own GraphQL type); reuse lint compliant.

Not addressed here, and deliberately so: pushed_predicates remains
Vec<String> built by format!("{:?}", c), so conditions still cross the
boundary as Rust Debug output rather than as ConditionKind. Retyping it is a
larger change with its own serialisation surface, and it is not needed to stop
the cost model lying.

`PlanNode` has four fields; `PlanStep` had six, and neither `projections` nor
`early_limit` was among them. The optimizer computed both, then dropped them at
the logical -> physical boundary, so the executor never saw either pushdown.

That is worse than merely lossy, and this is the part worth flagging.
`CostModel::estimate` already *discounts* a node carrying an early limit
(`cost.rs:345-350`):

    if let Some(limit) = node.early_limit {
        let limit_factor = (limit as f64 / 1000.0).min(1.0);
        selectivity *= limit_factor;
        time_ms *= 0.5 + 0.5 * limit_factor; // At least 50% of base cost
    }

So a plan was priced as cheap as half its base cost *because of* a limit
pushdown that the physical plan had no way to perform. The planner was
rewarding an optimisation it then discarded. Nothing failed loudly — plans
simply cost less on paper than they cost to run, which is the kind of defect
that shows up as unexplained latency rather than as an error.

Measured before the change: `early_limit` had exactly one non-test read in the
entire tree — the cost adjustment above. `projections` had **none**; nothing
read it after `vcl_bridge.rs` built it.

Both fields are now on `PlanStep` and populated from the node. They carry
`#[serde(default)]`, so a `PhysicalPlan` serialised before this change still
deserialises (absent -> `vec![]` / `None`, matching the old behaviour rather
than inventing a limit).

This surfaced while discharging `optimize_is_permutation` (#206). The Coq model
abstracts a node as `(modality, list condition)` — precisely the projection that
omits the two fields being lost. The abstraction was not wrong, but it was drawn
around the bug, which is why proving the permutation property could not have
caught it.

Test: `test_pushdown_survives_optimization` asserts both fields survive, and
matches steps by modality rather than index — asserting on `steps[0]` would test
the sort order, not the pushdown. Verified as a real guard, not a vacuous one:
with the population reverted it fails with `projection pushdown was dropped at
the physical boundary`; with the fix it passes.

Verified: `cargo test -p verisim-planner` 118 passed / 0 failed; `cargo clippy
-p verisim-planner --all-targets` clean; `cargo check -p verisim-api
--all-targets` clean (it maps `PlanStep` into its own GraphQL type); `reuse
lint` compliant.

Not addressed here, and deliberately so: `pushed_predicates` remains
`Vec<String>` built by `format!("{:?}", c)`, so conditions still cross the
boundary as Rust `Debug` output rather than as `ConditionKind`. Retyping it is a
larger change with its own serialisation surface, and it is not needed to stop
the cost model lying.
@gitar-bot

gitar-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

Note

Automatic reviews are paused because your trial's included automatic processing has been used for this period. Upgrade now, or comment "Gitar review" to run a review anytime.
Learn more

Code Review ✅ Approved

Carries projections and early_limit into the physical plan to fix silent pushdown drops during the logical-to-physical translation. No issues found.

Auto-approved and auto-merge armed: No blocking issues found.
Please see Auto-approve Docs for details on setting custom approval criteria. — merges when pipeline and required approvals pass.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@gitar-bot

gitar-bot Bot commented Jul 28, 2026

Copy link
Copy Markdown

⚠️ Gitar auto-approved this PR but could not enable auto-merge: auto-merge is disabled for this repository — enable "Allow auto-merge" in the repository settings.

@gitar-bot gitar-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gitar has auto-approved this PR and enabled auto-merge (configure)

@gitar-bot gitar-bot Bot added the gitar-approved Added by Gitar label Jul 28, 2026
`cargo doc` is a required check and `[`CostModel::estimate`]` does not resolve
from `plan.rs` — the type lives in the sibling `cost` module and is not in
scope there, so rustdoc failed the workspace build with "unresolved link".

Qualified to `crate::cost::CostModel::estimate`. Verified with the same command
CI runs (`cargo doc --no-deps`), plus `cargo test -p verisim-planner` still
green.
@hyperpolymath
hyperpolymath merged commit bc1f194 into main Jul 28, 2026
41 checks passed
@hyperpolymath
hyperpolymath deleted the fix/planner-carry-projections branch July 28, 2026 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gitar-approved Added by Gitar

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant