Skip to content

fix keycloak missing resource error check#69028

Open
stephen-bracken wants to merge 5 commits into
apache:mainfrom
stephen-bracken:fix-keycloak-error
Open

fix keycloak missing resource error check#69028
stephen-bracken wants to merge 5 commits into
apache:mainfrom
stephen-bracken:fix-keycloak-error

Conversation

@stephen-bracken

@stephen-bracken stephen-bracken commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Fix Auth manager check for keycloak 'resource not found' error

The error raised by keycloak that should be handled is {"error":"invalid_resource", "error_description": "Resource with id [Dag:team-a] does not exist."}

Sample log:

{"timestamp":"2026-06-26T10:37:18.612975Z","level":"error","event":"Exception in ASGI application\n","logger":"uvicorn.error","filename":"httptools_impl.py","lineno":425,"exception":[{"exc_type":"AirflowException","exc_value":"Request not recognized by Keycloak. invalid_resource. Resource with id [Dag:<team name>] does not exist.","exc_notes":[],"syntax_error":null,"is_cause":false,"frames":[{"filename":"/home/airflow/.local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py","lineno":420,"name":"run_asgi"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py","lineno":60,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/applications.py","lineno":1159,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/applications.py","lineno":107,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/errors.py","lineno":186,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/errors.py","lineno":164,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/common/http_access_log.py","lineno":83,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/gzip.py","lineno":29,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/gzip.py","lineno":130,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/gzip.py","lineno":46,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/base.py","lineno":191,"name":"__call__"},{"filename":"/usr/python/lib/python3.11/contextlib.py","lineno":158,"name":"__exit__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_utils.py","lineno":87,"name":"collapse_excgroups"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/base.py","lineno":193,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py","lineno":61,"name":"dispatch"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/base.py","lineno":168,"name":"call_next"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/base.py","lineno":144,"name":"coro"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py","lineno":63,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_exception_handler.py","lineno":53,"name":"wrapped_app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_exception_handler.py","lineno":42,"name":"wrapped_app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py","lineno":18,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/routing.py","lineno":716,"name":"__call__"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/routing.py","lineno":736,"name":"app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/routing.py","lineno":290,"name":"handle"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/routing.py","lineno":134,"name":"app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_exception_handler.py","lineno":53,"name":"wrapped_app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/_exception_handler.py","lineno":42,"name":"wrapped_app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/routing.py","lineno":120,"name":"app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/routing.py","lineno":457,"name":"app"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/fastapi/dependencies/utils.py","lineno":680,"name":"solve_dependencies"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/starlette/concurrency.py","lineno":32,"name":"run_in_threadpool"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/anyio/to_thread.py","lineno":63,"name":"run_sync"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py","lineno":2518,"name":"run_sync_in_worker_thread"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py","lineno":1002,"name":"run"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/core_api/security.py","lineno":262,"name":"depends_permitted_dags_filter"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/utils/session.py","lineno":100,"name":"wrapper"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py","lineno":578,"name":"get_authorized_dag_ids"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py","lineno":455,"name":"filter_authorized_dag_ids"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/cache.py","lineno":77,"name":"single_flight"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py","lineno":453,"name":"query_keycloak"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py","lineno":612,"name":"filter_authorized_dag_ids"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py","lineno":612,"name":"<setcomp>"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py","lineno":608,"name":"_is_authorized_dag_id"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py","lineno":229,"name":"is_authorized_dag"},{"filename":"/home/airflow/.local/lib/python3.11/site-packages/airflow/providers/keycloak/auth_manager/keycloak_auth_manager.py","lineno":434,"name":"_is_authorized"}],"is_group":false,"exceptions":[]}]}

related: #68943
related: #68951

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
  • No

@stephen-bracken stephen-bracken marked this pull request as ready for review June 26, 2026 14:06
@vincbeck

Copy link
Copy Markdown
Contributor

Tagging @onlyarnav here, you are modifying the constant KEYCLOAK_RESOURCE_NOT_FOUND_ERROR that @onlyarnav uses to match exception thrown by Keycloak when a resource does not exist. It might break things. @onlyarnav, can you review this one please?

@stephen-bracken

Copy link
Copy Markdown
Contributor Author

@vincbeck Updating the constant should be fine as its only used with this function

@stephen-bracken

Copy link
Copy Markdown
Contributor Author

@vincbeck @onlyarnav I've updated the PR to check against the error field from the keycloak json response for invalid_resource in the same way that the 400 response is handled here:

error = json.loads(resp.text)
raise AirflowException(
f"Request not recognized by Keycloak. {error.get('error')}. {error.get('error_description')}"
)

@vincbeck

Copy link
Copy Markdown
Contributor

You really seem to override what @onlyarnav did, I'd like him to review it

@onlyarnav

Copy link
Copy Markdown
Contributor

Thanks for the update! I just had a look at it.

My PR handled the error based on the original issue, where the response appeared to be a plain text string (resource not found: ...) rather than a JSON response. Since @stephen-bracken reported the original issue, I only had the information available there and wasn't aware that Keycloak could also return the structured invalid_resource JSON response.
Now that you shared the sample log, it's much clear.

From what you've shared, checking the structured JSON response via error == "invalid_resource" looks like a cleaner approach, and it aligns with the log you've posted.

My only question is whether the original plain-text resource not found: ... response is still possible on any supported Keycloak version or configuration. If that format can still occur, it might be worth handling both cases to avoid a regression. Otherwise, I'm happy with the updated approach.

onlyarnav

This comment was marked as outdated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants