Add XCom read access to callback supervisor comms channel#66611
Add XCom read access to callback supervisor comms channel#66611seanghaeli wants to merge 3 commits into
Conversation
|
What is the plan for plumbing the dag_id, run_id, etc through here? You don't seem to have addressed thr "motivation" part of your description. The reason I didn't include this one in the initial batch of comms channels was because GetXCom requires those and I didn't want users to have to manually provide those like |
|
Good question. The plan is for PR #66608 to provide the context plumbing — once a callback has access to So this PR gives the mechanism (XCom comms channel), and #66608 gives the identifiers. Once #66608 lands, we can add a convenience wrapper that pre-fills |
90f3323 to
91e5a68
Compare
4a33ab3 to
454244a
Compare
cfdf5dc to
94b8c53
Compare
Accept workload tokens on the get_xcom, connections, and variables execution API routes so a deadline callback subprocess can read XCom values (and connections/variables) via the supervisor comms channel, mirroring task access.
94b8c53 to
cccbd89
Compare
This PR's callback comms only use GetDagRun/GetConnection/GetVariable, never GetXCom, so the token:workload scope changes on the XCom execution-API route (and their TestWorkloadTokenScopeEnforcement tests) had no consumer here — they are apache#66611's deliverable (callback XCom read access) and were riding along. Revert xcoms.py and test_xcoms.py to main so this PR stays scoped to callback context fetching; apache#66611 owns the XCom route + comms changes.
This PR's callback comms only use GetDagRun/GetConnection/GetVariable, never GetXCom, so the token:workload scope changes on the XCom execution-API route (and their TestWorkloadTokenScopeEnforcement tests) had no consumer here — they are apache#66611's deliverable (callback XCom read access) and were riding along. Revert xcoms.py and test_xcoms.py to main so this PR stays scoped to callback context fetching; apache#66611 owns the XCom route + comms changes.
Callback supervisors carry a workload-scoped JWT, but the XCom read routes
only accepted execution-scoped tokens, so a callback's XCom read got 403
("Token type 'workload' not allowed"). Mirror the variables/connections
routes: set route_class=ExecutionAPIRoute on the router (so per-route
token:* scopes are honored) and add token:workload to the GET item/slice
and HEAD routes alongside the existing get_xcom route. POST/DELETE stay
execution-only.
Add read-only XCom access to the callback supervisor IPC channel, allowing
callbacks to fetch XCom values from upstream tasks via the same mechanism
used for Connections and Variables.
Motivation
PR #65269 added Connection, Variable, and MaskSecret comms channels to the
callback supervisor. XCom was deferred because it requires explicit dag_id,
run_id, and task_id (callbacks have no implicit task context). This PR adds
that missing piece.
Changes
GetXComto theCallbackToSupervisorunion typehandle_get_xcomshared handler inrequest_handlers.pyGetXCommessages inCallbackSubprocess._handle_requestOnly read access (GetXCom) is exposed. SetXCom and DeleteXCom remain out of
scope for callbacks since they should not be producing XCom values.
related: #65269
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.6) following the guidelines