Skip to content

Thread version_data to callbacks#69185

Merged
o-nikolas merged 4 commits into
apache:mainfrom
aws-mwaa:onikolas/pr3/s3-bundle-version-callbacks
Jul 7, 2026
Merged

Thread version_data to callbacks#69185
o-nikolas merged 4 commits into
apache:mainfrom
aws-mwaa:onikolas/pr3/s3-bundle-version-callbacks

Conversation

@o-nikolas

@o-nikolas o-nikolas commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

PR #67217 wired version_data into the task execution path but not callbacks, so callbacks for a pinned run initialized their bundle without the manifest tasks used. Populate version_data on the callback producer paths (ExecuteCallback.make and the CallbackRequest creation sites) under the same pin guard as tasks, carry it on BaseCallbackRequest, and forward it through prepare_callback_bundle to get_bundle.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@boring-cyborg boring-cyborg Bot added area:DAG-processing area:Executors-core LocalExecutor & SequentialExecutor area:Scheduler including HA (high availability) scheduler area:Triggerer labels Jun 30, 2026
@o-nikolas
o-nikolas force-pushed the onikolas/pr3/s3-bundle-version-callbacks branch from e3bace1 to a9cf234 Compare June 30, 2026 16:39

@SameerMesiah97 SameerMesiah97 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits that you can choose to address. Otherwise, this looks good to me.

Comment thread airflow-core/src/airflow/callbacks/callback_requests.py Outdated
Comment thread airflow-core/src/airflow/models/dag_version.py Outdated
Comment thread airflow-core/src/airflow/models/dag_version.py Outdated
Comment thread airflow-core/tests/unit/dag_processing/test_manager.py
Comment thread airflow-core/src/airflow/models/trigger.py
Comment thread airflow-core/src/airflow/models/trigger.py

@ramitkataria ramitkataria left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Besides the already open threads, looks good to me!

o-nikolas added 4 commits July 6, 2026 16:11
PR apache#67217 wired version_data into the task execution path but not
callbacks, so callbacks for a pinned run initialized their bundle
without the manifest tasks used. Populate version_data on the
callback producer paths (ExecuteCallback.make and the
CallbackRequest creation sites) under the same pin guard as tasks,
carry it on BaseCallbackRequest, and forward it through
prepare_callback_bundle to get_bundle.
@o-nikolas
o-nikolas force-pushed the onikolas/pr3/s3-bundle-version-callbacks branch from 91bdcf5 to bb04f5e Compare July 7, 2026 00:48

@ferruzzi ferruzzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like all concerns have been addressed.

@o-nikolas
o-nikolas merged commit 101bc15 into apache:main Jul 7, 2026
79 checks passed
@o-nikolas
o-nikolas deleted the onikolas/pr3/s3-bundle-version-callbacks branch July 7, 2026 22:35
@vatsrahul1001 vatsrahul1001 added this to the Airflow 3.4.0 milestone Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:DAG-processing area:Executors-core LocalExecutor & SequentialExecutor area:Scheduler including HA (high availability) scheduler area:Triggerer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants