Description
The Kube Watcher in the K8s executor keeps track of tasks that are intended to be launched as K8s pods. If something happens to these pods, such as they fail to launch before the pending timeout, the Kube Watcher marks the task as failed. Instead of just recording this in the scheduler log, as currently happens, it would be beneficial to write a log of what happened in the same location that the task log would have gone.
Use case/motivation
If something happens to a task pod, the Airflow UI doesn' show what happened to it at all. The task logs are completely blank. One needs to check the scheduler logs to troubleshoot. But in some organizations, not everyone who has access to the task logs via the Airflow UI also has access to the scheduler logs. Also, the scheduler logs have a lot going on in them for someone who just wants to know why their task didn't run.
Related issues
#27758 Lays the groundwork for there to be contributors to task logs besides the worker
Are you willing to submit a PR?
Code of Conduct
Description
The Kube Watcher in the K8s executor keeps track of tasks that are intended to be launched as K8s pods. If something happens to these pods, such as they fail to launch before the pending timeout, the Kube Watcher marks the task as failed. Instead of just recording this in the scheduler log, as currently happens, it would be beneficial to write a log of what happened in the same location that the task log would have gone.
Use case/motivation
If something happens to a task pod, the Airflow UI doesn' show what happened to it at all. The task logs are completely blank. One needs to check the scheduler logs to troubleshoot. But in some organizations, not everyone who has access to the task logs via the Airflow UI also has access to the scheduler logs. Also, the scheduler logs have a lot going on in them for someone who just wants to know why their task didn't run.
Related issues
#27758 Lays the groundwork for there to be contributors to task logs besides the worker
Are you willing to submit a PR?
Code of Conduct