Official Helm Chart version
1.3.0 (latest released)
Apache Airflow version
2.2.1
Kubernetes Version
1.21
Helm Chart configuration
enableBuiltInSecretEnvVars:
AIRFLOW__CORE__SQL_ALCHEMY_CONN: false
config:
secrets:
backend: airflow.providers.hashicorp.secrets.vault.VaultBackend
backend_kwargs: '{"mount_point": "...","url": "..."}'
Docker Image customisations
Nothing fancy, just adding some ACs and pip install hashicorp provider
What happened
I decided to use the new chart feature, letting us disable the basic key (AIRFLOW__CORE__SQL_ALCHEMY_CONN in my example) to use AIRFLOW__CORE__SQL_ALCHEMY_CON_SECRET from a secret backend.
Worked fine on both jobs (create-user & run-for-migration) but failed on every pod initContainer wait-for-migration
What you expected to happen
No response
How to reproduce
pod initContainers being able to retrieve conn_uri from secretBackend and pods job do
Anything else
After a debug session with @potiuk we found out the principal issue: the init containers in _helpers doesn't mount the airflow.cfg configMap: initContainers can't be aware of the secretBackend to retrieve the conn_uri because of this.
I would also like to point out the fact that an 'extraVolumes' and 'extraVolumeMounts' on global scope, like 'extraEnvs' exists, would be really really nice to have: the secretbackend usage showed me the necessity to mount secrets containing ACs as volumeMounts: to be able to propagate this, i had to modify every single /templates/*-deployment.yaml to add them manually.
=> every chart update would require to change this again and again
Regards & cheers !
Are you willing to submit PR?
Code of Conduct
Official Helm Chart version
1.3.0 (latest released)
Apache Airflow version
2.2.1
Kubernetes Version
1.21
Helm Chart configuration
Docker Image customisations
Nothing fancy, just adding some ACs and pip install hashicorp provider
What happened
I decided to use the new chart feature, letting us disable the basic key (
AIRFLOW__CORE__SQL_ALCHEMY_CONNin my example) to useAIRFLOW__CORE__SQL_ALCHEMY_CON_SECRETfrom a secret backend.Worked fine on both jobs (create-user & run-for-migration) but failed on every pod initContainer wait-for-migration
What you expected to happen
No response
How to reproduce
pod initContainers being able to retrieve conn_uri from secretBackend and pods job do
Anything else
After a debug session with @potiuk we found out the principal issue: the init containers in _helpers doesn't mount the airflow.cfg configMap: initContainers can't be aware of the secretBackend to retrieve the conn_uri because of this.
I would also like to point out the fact that an '
extraVolumes' and 'extraVolumeMounts' on global scope, like 'extraEnvs' exists, would be really really nice to have: the secretbackend usage showed me the necessity to mount secrets containing ACs as volumeMounts: to be able to propagate this, i had to modify every single /templates/*-deployment.yaml to add them manually.=> every chart update would require to change this again and again
Regards & cheers !
Are you willing to submit PR?
Code of Conduct