Skip to content

Airflow 2.2.0 - catchup=False not working #19126

Description

@andormarkus

Apache Airflow version

2.2.0 (latest released)

Operating System

macOS 11.5.2 or Amazon Linux 2 on EKS

Versions of Apache Airflow Providers

No response

Deployment

Docker-Compose

Deployment details

No response

What happened

it looks like In Airflow 2.2.0 catchup=False is ignored. The simple_dag is scheduled to run in ever minute.

I did not experience this behivour in Airflow 2.1.3

Airflow 2.2.0

I have paused the DAG at 11:31
Screen Shot 2021-10-21 at 11 31 51

I have unpaused the DAG at 11:35 and it executed all the skipped run despite catchup=False
Screen Shot 2021-10-21 at 11 35 56

Airflow 2.1.2

I have paused the DAG at 11:45
Screen Shot 2021-10-21 at 11 45 22

I have unpaused the DAG at 11:50 and it did not executed all the skipped run
Screen Shot 2021-10-21 at 11 50 27

What you expected to happen

All the DAG run which was scheduled to run under the pause period will not run after I unpause a DAG.

How to reproduce

  1. Use the official docker-compose file - https://airflow.apache.org/docs/apache-airflow/stable/docker-compose.yaml
  2. Add simple_dag.py
    """Sample DAG."""
    
    import time
    from datetime import datetime, timedelta
    
    from airflow import DAG
    from airflow.operators.python import PythonOperator
    
    default_args = {
        "owner": "airflow",
        "depends_on_past": False,
        "start_date": datetime(2020, 1, 1),
        "email": ["support@airflow.com"],
        "email_on_failure": False,
        "email_on_retry": False,
        "retries": 1,
        "retry_delay": timedelta(minutes=5),
    }
    
    
    def sleep() -> bool:
        time.sleep(10)
        return True
    
    
    with DAG("simple_dag", default_args=default_args, schedule_interval="* * * * *", catchup=False) as dag:
        t1 = PythonOperator(task_id="sleep", python_callable=sleep)
  3. Run simple_dag for few minutes
  4. Pause simple_dag
  5. Wait few minutes
  6. Unpause simple_dag

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions