Skip to content

Allow K8s Executor to add logs for tasks that failed to launch as a worker pod #29025

Description

@collinmcnulty

Description

The Kube Watcher in the K8s executor keeps track of tasks that are intended to be launched as K8s pods. If something happens to these pods, such as they fail to launch before the pending timeout, the Kube Watcher marks the task as failed. Instead of just recording this in the scheduler log, as currently happens, it would be beneficial to write a log of what happened in the same location that the task log would have gone.

Use case/motivation

If something happens to a task pod, the Airflow UI doesn' show what happened to it at all. The task logs are completely blank. One needs to check the scheduler logs to troubleshoot. But in some organizations, not everyone who has access to the task logs via the Airflow UI also has access to the scheduler logs. Also, the scheduler logs have a lot going on in them for someone who just wants to know why their task didn't run.

Related issues

#27758 Lays the groundwork for there to be contributors to task logs besides the worker

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions