Skip to content

Reconnect to blocked Databricks runs in SubmitRunOperator durable retries#69195

Merged
amoghrajesh merged 1 commit into
apache:mainfrom
astronomer:databricks-submit-run-continue
Jul 1, 2026
Merged

Reconnect to blocked Databricks runs in SubmitRunOperator durable retries#69195
amoghrajesh merged 1 commit into
apache:mainfrom
astronomer:databricks-submit-run-continue

Conversation

@amoghrajesh

@amoghrajesh amoghrajesh commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Similar to f0c3db4 but for the other databricks operator, built on: #69193

Why

With durable execution, on retry the operator inspects the stored run's state to decide whether to reconnect or resubmit. BLOCKED (gated by concurrency/dependencies) and WAITING_FOR_RETRY are in-flight states, but is_job_active didn't list them, so a durable retry treated a run that was merely waiting as not-active and submitted a duplicate. Both states are now recognized as active, so the retry reconnects and waits.


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@phanikumv phanikumv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch, adding BLOCKED and WAITING_FOR_RETRY makes is_job_active exactly the complement of RunState.is_terminal, so every in-flight state reconnects and polls to completion.

Coverage on both the parametrized cases and the end-to-end BLOCKED reconnect looks good.

@amoghrajesh

Copy link
Copy Markdown
Contributor Author

Thank you. Since its a simple one, merging.

@amoghrajesh
amoghrajesh merged commit 579aa9f into apache:main Jul 1, 2026
81 checks passed
@amoghrajesh
amoghrajesh deleted the databricks-submit-run-continue branch July 1, 2026 08:47
@github-project-automation github-project-automation Bot moved this from In progress to Done in Durable / Crash-Safe Execution Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

2 participants