"Not a valid timetable" when returning None from next_dagrun_info in a custom timetable

[nicogelders]

Apache Airflow version

2.2.0 (latest released)

Operating System

Mac

Versions of Apache Airflow Providers

No response

Deployment

Other Docker-based deployment

Deployment details

No response

What happened

Getting an exception when returning None from next_dagrun_info in a custom timetable. The timetable protocol says that when None is returned a DagRun will not happen but right now it throws an exception.

Exception:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line 623, in _execute
    self._run_scheduler_loop()
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line 704, in _run_scheduler_loop
    num_queued_tis = self._do_scheduling(session)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line 787, in _do_scheduling
    callback_to_run = self._schedule_dag_run(dag_run, session)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line 1039, in _schedule_dag_run
    self._update_dag_next_dagruns(dag, dag_model, active_runs)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/jobs/scheduler_job.py", line 930, in _update_dag_next_dagruns
    data_interval = dag.get_next_data_interval(dag_model)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/dag.py", line 629, in get_next_data_interval
    return self.infer_automated_data_interval(dag_model.next_dagrun)
  File "/home/airflow/.local/lib/python3.9/site-packages/airflow/models/dag.py", line 671, in infer_automated_data_interval
    raise ValueError(f"Not a valid timetable: {self.timetable!r}")
ValueError: Not a valid timetable: <my_timetables.workday_timetable.WorkdayTimetable object at 0x7f42b1f02430>

What you expected to happen

DagRun to not happen.

How to reproduce

Create a custom timetable and return None in next_dagrun_info after a few DagRuns are created by that timetable

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[uranusjr]

(Copying the conversation on Slack)

The cause is when the scheduler sees data_interval=None it goes "oh this doesn't have a data interval, so if it's scheduled it must have been done before 2.2". But it forgot to check the "if it's scheduled" part and assumes it is.

So DAG.get_next_data_interval() should check DagModel.next_dagrun first, and only try to get an automated data interval if that value is not None (i.e. if there is actually a next DAG run being scheduled).