Cleanup Job Pods host remove history after completing the job

[potiuk]

We currently do not have possibility of confioguring:

• spec.successfulJobsHistoryLimit
• spec.failedJobsHistoryLimit

For cleanup jobs. Creaeted it to add a feature for that.

Discussed in #26547

^{Originally posted by alionar September 21, 2022}

Airflow Version: 2.2.2
Kubernetes Version: 1.22.12-gke.500
Helm Chart version: 1.6.0

Hi, i found that completed cleanup job pods just stayed in nodes after finished and make GKE auto scaller triggered to add new nodes everytime cleanup job pods executed.

It made us to delete all completed job pods and drain the unused nodes manually everyday.

I found in k8s docs that:

The .spec.successfulJobsHistoryLimit and .spec.failedJobsHistoryLimit fields are optional. 
These fields specify how many completed and failed jobs should be kept. 
By default, they are set to 3 and 1 respectively

But when i tried to add this history limit as 0 into cleanup helm chart value, chart didn't recognize it

is that possible to add some job history limit number in cleanup value chart?

Current values.yaml for cleanup:

cleanup:
  enabled: true
  schedule: "*/15 * * * *"

[MatthieuBlais]

I can work on that