Fix custom timetables silently failing to schedule on Airflow 3.2 - #68326
Closed
Abdulrehman-PIAIC80387 wants to merge 3 commits into
Closed
Fix custom timetables silently failing to schedule on Airflow 3.2#68326Abdulrehman-PIAIC80387 wants to merge 3 commits into
Abdulrehman-PIAIC80387 wants to merge 3 commits into
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
jroachgolf84
suggested changes
Jun 11, 2026
Abdulrehman-PIAIC80387
force-pushed
the
fix-custom-timetable-silent-scheduling-stop
branch
from
June 12, 2026 06:20
cbdbaba to
47245e5
Compare
Author
|
Hi @jroachgolf84, thanks for the review! All comments are resolved in commit 47245e5. Any further feedback is welcome! |
Lee-W
reviewed
Jun 12, 2026
Lee-W
reviewed
Jun 12, 2026
Lee-W
reviewed
Jun 12, 2026
Member
|
#68342 is closer to being merged. So I'm going to merge that first. if you're still interested in it, we can extend the test case in the the parameterization way I mentioned |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Airflow 3.2 extended
DagRunInfowith two new fields for partition-oriented scheduling (AIP-76, #61167):Both were added without defaults, making them required positional fields. Custom timetables written for 3.1.x and earlier — including the pattern shown in the official custom-timetable how-to — construct
DagRunInfowith only two arguments:On 3.2 this raises
TypeError: DagRunInfo.__new__() missing 2 required positional arguments: 'partition_date' and 'partition_key'. The exception is swallowed by the error handling aroundDAG.next_dagrun_info(), sonext_dagrun/next_dagrun_create_afterstayNULL, the Next Run column is empty, and no scheduled runs are created — with no import error or other visible failure. Built-in timetables are unaffected because they buildDagRunInfovia helpers (DagRunInfo.interval(...),DagRunInfo.exact(...), etc.) that already pass the partition fields explicitly.Fix
Default the two partition fields to
None. They are the trailing fields of theNamedTuple, so the change is purely additive and restores the documented two-argument construction. Every internal call site already passes all four fields explicitly, so their behaviour is unchanged.Design notes
NamedTuplefields is the idiomatic approach already used in this codebase — e.g.TaskInstanceKey(try_number: int = 1,map_index: int = -1). No__new__override or dataclass conversion is needed.partition_date/partition_keyexplicitly; the defaults only affect callers that omit them, which is exactly the pre-3.2 custom-timetable contract.Tests
Added two unit tests in
test_base_timetable.py: the backward-compatible two-argument construction (partition fields default toNone) and the explicit four-argument construction (partition fields still settable). Fullairflow-core/tests/unit/timetables/suite passes (220 passed).closes: #68315