From 24fddfe2695c70961b0ea03e3e2f9f5c631f35fa Mon Sep 17 00:00:00 2001 From: Kaxil Naik Date: Fri, 10 Apr 2026 23:42:29 +0100 Subject: [PATCH 1/2] Add FAQ entry for API server memory growth with gunicorn fix Users experiencing API server memory growth from accumulated serialized DAG cache entries can use gunicorn with rolling worker restarts as the recommended solution. This FAQ entry documents the configuration and cross-references the dag version inflation FAQ. --- airflow-core/docs/faq.rst | 37 +++++++++++++++++++++++++++++++++++++ 1 file changed, 37 insertions(+) diff --git a/airflow-core/docs/faq.rst b/airflow-core/docs/faq.rst index fe48c3695dc81..04a65ee399eed 100644 --- a/airflow-core/docs/faq.rst +++ b/airflow-core/docs/faq.rst @@ -666,6 +666,43 @@ try pausing the Dag again, or check the console or server logs if the issue recurs. +API Server +^^^^^^^^^^ + +.. _faq:api-server-memory-growth: + +How to prevent API server memory growth? +----------------------------------------- + +The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate +(see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory. + +The recommended solution is to use **gunicorn** with **rolling worker restarts**. Gunicorn periodically +recycles worker processes, releasing all accumulated memory. It also uses ``preload`` + ``fork``, so +workers share read-only memory pages via copy-on-write, reducing overall memory usage by 40-50% compared +to uvicorn's multiprocess mode. + +To enable gunicorn with worker recycling: + +.. code-block:: ini + + [api] + server_type = gunicorn + # Restart each worker every 12 hours (43200 seconds) + worker_refresh_interval = 43200 + worker_refresh_batch_size = 1 + +This requires the ``apache-airflow-core[gunicorn]`` extra to be installed. + +See :ref:`config:api__server_type`, :ref:`config:api__worker_refresh_interval`, and +:ref:`config:api__worker_refresh_batch_size` for the full configuration reference. + +.. note:: + + Worker recycling handles memory growth from *any* source, not just the Dag cache. It is the + recommended approach for production API server deployments. + + MySQL and MySQL variant Databases ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From 1f9f130e0d689c416e1adae3354b8d0d8ecba10a Mon Sep 17 00:00:00 2001 From: Kaxil Naik Date: Sat, 11 Apr 2026 00:20:16 +0100 Subject: [PATCH 2/2] Add version availability note for gunicorn support --- airflow-core/docs/faq.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/airflow-core/docs/faq.rst b/airflow-core/docs/faq.rst index 04a65ee399eed..a5064cc3dfab4 100644 --- a/airflow-core/docs/faq.rst +++ b/airflow-core/docs/faq.rst @@ -677,10 +677,10 @@ How to prevent API server memory growth? The API server caches serialized Dag objects in memory. Over time, as Dag versions accumulate (see :ref:`faq:dag-version-inflation`), this cache grows and can consume several gigabytes of memory. -The recommended solution is to use **gunicorn** with **rolling worker restarts**. Gunicorn periodically -recycles worker processes, releasing all accumulated memory. It also uses ``preload`` + ``fork``, so -workers share read-only memory pages via copy-on-write, reducing overall memory usage by 40-50% compared -to uvicorn's multiprocess mode. +The recommended solution (available since Airflow 3.2.0) is to use **gunicorn** with **rolling worker +restarts**. Gunicorn periodically recycles worker processes, releasing all accumulated memory. It also +uses ``preload`` + ``fork``, so workers share read-only memory pages via copy-on-write, reducing overall +memory usage by 40-50% compared to uvicorn's multiprocess mode. To enable gunicorn with worker recycling: