Skip to content

Fix backfill validate n plus one#66313

Draft
ColtenOuO wants to merge 2 commits into
apache:mainfrom
ColtenOuO:fix-backfill-validate-n-plus-one
Draft

Fix backfill validate n plus one#66313
ColtenOuO wants to merge 2 commits into
apache:mainfrom
ColtenOuO:fix-backfill-validate-n-plus-one

Conversation

@ColtenOuO

@ColtenOuO ColtenOuO commented May 3, 2026

Copy link
Copy Markdown
Contributor

Summary

A new helper _fetch_latest_dag_runs_for_infos pre-loads all DagRuns the loop will need into a dict, replacing the per-info SELECT inside the loop with an in-memory lookup. This collapses N database round-trips into one batched SELECT plus N dict reads.

Per @jason810496's review on #66222, a regression test (test_create_backfill_does_not_re_query_each_info) uses assert_queries_count to lock in the post-fix query count.

Impact

  • 20-day backfill: 215 → 195 round-trips (one saved per DagRunInfo).

Verification

  • Full backfill suite: 60 passed in airflow-core/tests/unit/models/test_backfill.py.
  • API-layer suite: 39 passed in airflow-core/tests/unit/api_fastapi/core_api/routes/public/test_backfills.py.

Was generative AI tooling used to co-author this PR?
  • Yes (Claude Code)

Generated-by: Claude Code following the guidelines


Important

🛠️ Maintainer triage note for @ColtenOuO · by @potiuk · 2026-06-18 13:57 UTC

Paused pending your next update — this PR has been inactive for ~43 days, so it's been moved to draft to keep the review queue clear:

  • Rebase on the latest main, address any new failures, and mark it Ready for review when you pick it back up — no rush.
  • See the Pull Request quality criteria.

The ball is in your court — you've been assigned to this PR.

Automated triage — may be imperfect; a maintainer takes the next look.

ColtenOuO added 2 commits May 3, 2026 16:20
Signed-off-by: ColtenOuO <jun930436@gmail.com>
Signed-off-by: ColtenOuO <jun930436@gmail.com>
@ColtenOuO ColtenOuO requested review from XD-DENG and ashb as code owners May 3, 2026 17:03
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 5, 2026
@potiuk potiuk marked this pull request as draft June 18, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants