Skip to content

Fix circular import between connection and configuration modules#63787

Merged
potiuk merged 1 commit intoapache:mainfrom
Dev-iL:2603/fix-connection-circular-import
Mar 19, 2026
Merged

Fix circular import between connection and configuration modules#63787
potiuk merged 1 commit intoapache:mainfrom
Dev-iL:2603/fix-connection-circular-import

Conversation

@Dev-iL
Copy link
Collaborator

@Dev-iL Dev-iL commented Mar 17, 2026

Summary

  • Defer from airflow.configuration import conf, ensure_secrets_loaded from module-level to inside get_connection_from_secrets(), breaking a circular import cycle where airflow.models.Connectionairflow.configuration → secrets backend init → airflow.models.Connection.

Details

connection.py imports airflow.configuration at module level. During startup, airflow.configuration initializes secrets backends, which in turn import airflow.models.Connection. This creates a circular import that can cause ImportError: Module "airflow.models.connection" does not define a "Connection" when the import is re-entered before the Connection class is fully defined. The fix moves the import to the only method that uses it (get_connection_from_secrets), keeping runtime behavior identical while eliminating the import-time recursion.

related: #63520


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Opus 4.6 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Importing airflow.models.Connection pulled in airflow.configuration, and airflow.configuration initializes secrets backends at import time. That initialization itself does `from airflow.models import Connection`, so under Python 3.14 the lazy import path could re-enter `airflow.models.__getattr__ before connection.py` had finished defining the Connection class. The result was the ImportError: Module "airflow.models.connection" does not define a "Connection"; plus repeated SQLAlchemy warnings about redefining the model during partial imports. By deferring: `from airflow.configuration import conf, ensure_secrets_loaded` until the method actually needs them, importing the module no longer triggers that cycle. `Connection` gets fully defined first, and only later, when get_connection_from_secrets() runs, do we touch configuration and secrets loading. That keeps behavior the same at runtime, but removes the import-time recursion.
@Dev-iL Dev-iL requested review from XD-DENG and ashb as code owners March 17, 2026 07:38
@Dev-iL Dev-iL mentioned this pull request Mar 17, 2026
6 tasks
@eladkal eladkal requested a review from uranusjr March 17, 2026 20:51
@eladkal eladkal added this to the Airflow 3.1.9 milestone Mar 19, 2026
@uranusjr uranusjr added the backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch label Mar 19, 2026
@potiuk potiuk merged commit c36ce1b into apache:main Mar 19, 2026
75 checks passed
@github-actions
Copy link

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-1-test Commit Link

You can attempt to backport this manually by running:

cherry_picker c36ce1b v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@Dev-iL Dev-iL deleted the 2603/fix-connection-circular-import branch March 19, 2026 10:49
fat-catTW pushed a commit to fat-catTW/airflow that referenced this pull request Mar 22, 2026
Importing airflow.models.Connection pulled in airflow.configuration, and airflow.configuration initializes secrets backends at import time. That initialization itself does `from airflow.models import Connection`, so under Python 3.14 the lazy import path could re-enter `airflow.models.__getattr__ before connection.py` had finished defining the Connection class. The result was the ImportError: Module "airflow.models.connection" does not define a "Connection"; plus repeated SQLAlchemy warnings about redefining the model during partial imports. By deferring: `from airflow.configuration import conf, ensure_secrets_loaded` until the method actually needs them, importing the module no longer triggers that cycle. `Connection` gets fully defined first, and only later, when get_connection_from_secrets() runs, do we touch configuration and secrets loading. That keeps behavior the same at runtime, but removes the import-time recursion.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-to-v3-1-test Mark PR with this label to backport to v3-1-test branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants