Skip to content

Mask secrets properly when using deprecated import path#58662

Merged
amoghrajesh merged 1 commit into
apache:mainfrom
amoghrajesh:compat-shim-for-masker-proper
Nov 26, 2025
Merged

Mask secrets properly when using deprecated import path#58662
amoghrajesh merged 1 commit into
apache:mainfrom
amoghrajesh:compat-shim-for-masker-proper

Conversation

@amoghrajesh

Copy link
Copy Markdown
Contributor

The deprecated import path 'airflow.sdk.execution_time.secrets_masker' was redirecting mask_secret() to a single-process implementation that only masked secrets in the task subprocess. This caused secrets to not be masked in logs because the supervisor process (which writes logs) did not receive masking instructions through this message: https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/log.py#L244-L250

Users importing mask_secret from the old path would see their secrets appear unmasked in task logs despite calling mask_secret() successfully.

The compatibility shim now redirects mask_secret to the supervisor aware implementation in airflow.sdk.log that properly masks secrets in both the task and supervisor processes.

Testing

DAG used:

import sys
from time import sleep

from airflow.sdk import DAG, Variable
from airflow.providers.standard.operators.python import PythonOperator
from airflow.exceptions import AirflowTaskTimeout

from airflow.sdk.execution_time.secrets_masker import mask_secret

mask_secret("username")


def py_callable():
    print("Hello World!")
    print("This is an username: username")
    print(f"This is in dict form: {{'key': 'username'}}")
    print(f"This is in list form: {[1, 2, 3, 'username']}")


with DAG(
    dag_id="testing_secrets_masker",
    schedule=None,
    catchup=False,
    tags=["hello", "world"],
) as dag:
    my_task = PythonOperator(
        task_id="say_hello",
        python_callable=py_callable,
    )

Before the change:
image

After the change:

image

I also fixed the deprecation warning which was not being emitted.


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

@sunank200 sunank200 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall good change

Comment thread task-sdk/src/airflow/sdk/execution_time/secrets_masker.py
@amoghrajesh

Copy link
Copy Markdown
Contributor Author

I will try the fix for automatic backport on this PR: #58705

@amoghrajesh amoghrajesh merged commit 0afe6d3 into apache:main Nov 26, 2025
85 checks passed
github-actions Bot pushed a commit that referenced this pull request Nov 26, 2025
…58662)

(cherry picked from commit 0afe6d3)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Backport successfully created: v3-1-test

Status Branch Result
v3-1-test PR Link

github-actions Bot pushed a commit to aws-mwaa/upstream-to-airflow that referenced this pull request Nov 26, 2025
…pache#58662)

(cherry picked from commit 0afe6d3)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
potiuk pushed a commit that referenced this pull request Nov 28, 2025
…58662) (#58726)

(cherry picked from commit 0afe6d3)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
ephraimbuddy pushed a commit that referenced this pull request Dec 3, 2025
…58662) (#58726)

(cherry picked from commit 0afe6d3)

Co-authored-by: Amogh Desai <amoghrajesh1999@gmail.com>
Copilot AI pushed a commit to jason810496/airflow that referenced this pull request Dec 5, 2025
itayweb pushed a commit to itayweb/airflow that referenced this pull request Dec 6, 2025
Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Mar 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants