Skip to content

Enable native HPA for Airflow Workers#36174

Merged
potiuk merged 13 commits into
apache:mainfrom
pemiranda-clgx:enable-native-hpa
Jan 14, 2024
Merged

Enable native HPA for Airflow Workers#36174
potiuk merged 13 commits into
apache:mainfrom
pemiranda-clgx:enable-native-hpa

Conversation

@pemiranda-clgx

@pemiranda-clgx pemiranda-clgx commented Dec 11, 2023

Copy link
Copy Markdown

Add K8S native Horizontal Pod Autoscaler in Airflow Workers to support autoscaling based on external custom metrics.

@hussein-awala hussein-awala left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good; I just added a small comment about the chosen type for the HPA metrics.

Comment thread chart/values.schema.json Outdated
Comment thread chart/values.yaml Outdated
Comment thread chart/templates/workers/worker-hpa.yaml
Comment thread chart/values.yaml Outdated
Comment thread helm_tests/other/test_hpa.py
Comment thread helm_tests/other/test_hpa.py

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes @pemiranda-clgx
I have one concern regarding versioning.

It should be /v2 for K8S >= 1.23 and /v2beta2 for < 1.23

Comment thread chart/templates/workers/worker-hpa.yaml
@pemiranda-clgx pemiranda-clgx requested a review from ashb as a code owner January 3, 2024 21:31

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@potiuk potiuk merged commit 03cf692 into apache:main Jan 14, 2024
@ephraimbuddy ephraimbuddy added the changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..) label Feb 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:helm-chart Airflow Helm Chart changelog:skip Changes that should be skipped from the changelog (CI, tests, etc..)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants