From c97690e4bc104de5ddc91f58abfab8b8568d4028 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 15:25:38 -0500 Subject: [PATCH 01/18] Add section on built in timetables to timetables concept doc --- docs/apache-airflow/concepts/timetable.rst | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 9898204e99f11..ee1b994f65b86 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -48,3 +48,15 @@ the schedule. Some examples are: As such, Airflow allows for custom timetables to be written in plugins and used by DAGs. An example demonstrating a custom timetable can be found in the :doc:`/howto/timetable` how-to guide. + +Built In Timetables +------------------- +Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables +may be available in provider packages. + +* CronDataIntervalTimetable - Set timing based on a cron schedule. Can be selected by providing a string that is a valid + cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation. +* DeltaDataIntervalTimetable - Schedules data intervals with a time delta. Can be selected by providing a + ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the schedule_interval parameter of a DAG. +* EventsTimetable - Simply pass a list of datetimes for the DAG to run after. Useful for timing based on sporting + events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable. From dfd7d304bd55e2901387cfe1fee9c7f7922e6823 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 15:51:38 -0500 Subject: [PATCH 02/18] Add section on built in timetables to timetables concept doc --- docs/apache-airflow/concepts/timetable.rst | 44 ++++++++++++++++++++-- 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index ee1b994f65b86..85598d364ce38 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -54,9 +54,45 @@ Built In Timetables Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables may be available in provider packages. -* CronDataIntervalTimetable - Set timing based on a cron schedule. Can be selected by providing a string that is a valid +CronDataIntervalTimetable +------------------------- +Set timing based on a cron schedule. Can be selected by providing a string that is a valid cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation. -* DeltaDataIntervalTimetable - Schedules data intervals with a time delta. Can be selected by providing a + +.. code-block:: python + @dag( + schedule_interval="0 1 * * 3" # At 01:00 on Wednesday. + ... + ) + +DeltaDataIntervalTimetable +-------------------------- +Schedules data intervals with a time delta. Can be selected by providing a ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the schedule_interval parameter of a DAG. -* EventsTimetable - Simply pass a list of datetimes for the DAG to run after. Useful for timing based on sporting - events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable. + +.. code-block:: python + @dag( + schedule_interval=datetime.timedelta(minutes=30) # At 01:00 on Wednesday. + ... + ) + +EventsTimetable +--------------- +Simply pass a list of datetimes for the DAG to run after. Useful for timing based on sporting +events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable. + +The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally, +the ``restrict_to_events`` flag can be used to force manual runs of the DAG to use the time of the most recent (or very +first) event for the data interval, otherwise manual runs will run with a ``data_interval_start`` and +``data_interval_end`` equal to the time at which the manual run was begun. You can also name the set of events using the +``description`` parameter, which will be displayed in the Airflow UI. + +.. code-block:: python + @dag( + timetable=EventsTimetable(event_dates=[pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), + pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), + pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago")], + description="My Team's Baseball Games", + restrict_to_events=False) + ... + ) From 7e637279e825cec7b6be871cfb4ca8c8a6fc9186 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 16:07:54 -0500 Subject: [PATCH 03/18] Apply suggestions from code review Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> --- docs/apache-airflow/concepts/timetable.rst | 19 +++++++++++-------- 1 file changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 85598d364ce38..bed04c0bfd7c1 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -55,9 +55,10 @@ Airflow comes with several common timetables built in to cover the most common u may be available in provider packages. CronDataIntervalTimetable -------------------------- -Set timing based on a cron schedule. Can be selected by providing a string that is a valid - cron expression to the schedule_interval parameter of a DAG as described in the :doc:`/concepts/dags` documentation. +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Set schedule based on a cron expression. Can be selected by providing a string that is a valid + cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation. .. code-block:: python @dag( @@ -66,19 +67,21 @@ Set timing based on a cron schedule. Can be selected by providing a string that ) DeltaDataIntervalTimetable --------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^ + Schedules data intervals with a time delta. Can be selected by providing a - ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the schedule_interval parameter of a DAG. + ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. .. code-block:: python @dag( - schedule_interval=datetime.timedelta(minutes=30) # At 01:00 on Wednesday. + schedule_interval=datetime.timedelta(minutes=30) ... ) EventsTimetable ---------------- -Simply pass a list of datetimes for the DAG to run after. Useful for timing based on sporting +^^^^^^^^^^^^^^^ + +Simply pass a list of ``datetimes`` for the DAG to run after. Useful for timing based on sporting events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable. The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally, From c96f8a7e608d0648469b3ec66256a0db47f213ee Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 16:08:38 -0500 Subject: [PATCH 04/18] Correct spacing on timetables document --- docs/apache-airflow/concepts/timetable.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index bed04c0bfd7c1..8864810c20975 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -51,6 +51,7 @@ DAGs. An example demonstrating a custom timetable can be found in the Built In Timetables ------------------- + Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables may be available in provider packages. From 30ee7a9b2cb8172452d574b800d3be178886de96 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 17:44:47 -0500 Subject: [PATCH 05/18] Fix code block spacing on timetables.rst --- docs/apache-airflow/concepts/timetable.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 8864810c20975..ae8350db0de44 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -62,6 +62,7 @@ Set schedule based on a cron expression. Can be selected by providing a string t cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation. .. code-block:: python + @dag( schedule_interval="0 1 * * 3" # At 01:00 on Wednesday. ... @@ -74,6 +75,7 @@ Schedules data intervals with a time delta. Can be selected by providing a ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. .. code-block:: python + @dag( schedule_interval=datetime.timedelta(minutes=30) ... @@ -92,6 +94,7 @@ first) event for the data interval, otherwise manual runs will run with a ``data ``description`` parameter, which will be displayed in the Airflow UI. .. code-block:: python + @dag( timetable=EventsTimetable(event_dates=[pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), From 7682a1bda128c88996baff3698b6459024f4d34e Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 17:57:04 -0500 Subject: [PATCH 06/18] Review edits for timetables.rst --- docs/apache-airflow/concepts/timetable.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index ae8350db0de44..245611a576fe1 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -53,7 +53,7 @@ Built In Timetables ------------------- Airflow comes with several common timetables built in to cover the most common use cases. Additional timetables -may be available in provider packages. +may be available in plugins. CronDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -72,7 +72,7 @@ DeltaDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^^ Schedules data intervals with a time delta. Can be selected by providing a - ``datetime.timedelta`` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. + `:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. .. code-block:: python From 19b80a851424217ff31d859082142b416b5a1392 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 17:58:28 -0500 Subject: [PATCH 07/18] Apply formatting suggestions from code review Co-authored-by: Tzu-ping Chung --- docs/apache-airflow/concepts/timetable.rst | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 245611a576fe1..cccc3f581b749 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -84,7 +84,7 @@ Schedules data intervals with a time delta. Can be selected by providing a EventsTimetable ^^^^^^^^^^^^^^^ -Simply pass a list of ``datetimes`` for the DAG to run after. Useful for timing based on sporting +Simply pass a list of ``datetime``s for the DAG to run after. Useful for timing based on sporting events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable. The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally, @@ -96,10 +96,14 @@ first) event for the data interval, otherwise manual runs will run with a ``data .. code-block:: python @dag( - timetable=EventsTimetable(event_dates=[pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), - pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), - pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago")], - description="My Team's Baseball Games", - restrict_to_events=False) - ... + timetable=EventsTimetable( + event_dates=[ + pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), + pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), + pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago"), + ], + description="My Team's Baseball Games", + restrict_to_events=False, + ), + ..., ) From 79a88f259ac47cfa6f48970ec4d306a48d2d343f Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 18:03:10 -0500 Subject: [PATCH 08/18] Update docs/apache-airflow/concepts/timetable.rst Co-authored-by: Tzu-ping Chung --- docs/apache-airflow/concepts/timetable.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index cccc3f581b749..4e5e1378df865 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -72,7 +72,7 @@ DeltaDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^^ Schedules data intervals with a time delta. Can be selected by providing a - `:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. + :class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. .. code-block:: python From bca0b949f02bf0d00425290b9b848e2d753f7d8d Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Wed, 20 Apr 2022 09:30:23 -0500 Subject: [PATCH 09/18] Show import of EventsTimetable on documentation --- docs/apache-airflow/concepts/timetable.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 4e5e1378df865..107a72703a654 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -95,6 +95,8 @@ first) event for the data interval, otherwise manual runs will run with a ``data .. code-block:: python + from airflow.timetables.events import EventsTimetable + @dag( timetable=EventsTimetable( event_dates=[ From 8d08d9baff0726bfbdccf8e7945727d8e95992a9 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Wed, 29 Jun 2022 11:51:00 -0500 Subject: [PATCH 10/18] Simplify timetables example code blocks to pacify black --- docs/apache-airflow/concepts/timetable.rst | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 107a72703a654..d161879f2bb07 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -62,11 +62,7 @@ Set schedule based on a cron expression. Can be selected by providing a string t cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation. .. code-block:: python - - @dag( - schedule_interval="0 1 * * 3" # At 01:00 on Wednesday. - ... - ) + schedule_interval="0 1 * * 3" # At 01:00 on Wednesday. DeltaDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -76,10 +72,8 @@ Schedules data intervals with a time delta. Can be selected by providing a .. code-block:: python - @dag( - schedule_interval=datetime.timedelta(minutes=30) - ... - ) + schedule_interval=datetime.timedelta(minutes=30) + EventsTimetable ^^^^^^^^^^^^^^^ From 1bbba24db57de54679038b9c2a5460937b1be7c4 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 15:25:38 -0500 Subject: [PATCH 11/18] Add section on built in timetables to timetables concept doc --- docs/apache-airflow/concepts/timetable.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index d161879f2bb07..d13fa0595cd4e 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -103,3 +103,4 @@ first) event for the data interval, otherwise manual runs will run with a ``data ), ..., ) + From 1df6397b665c0259623de77e3f1563637028af3f Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 17:57:04 -0500 Subject: [PATCH 12/18] Review edits for timetables.rst --- docs/apache-airflow/concepts/timetable.rst | 35 +++++++++++----------- 1 file changed, 17 insertions(+), 18 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index d13fa0595cd4e..245611a576fe1 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -62,23 +62,29 @@ Set schedule based on a cron expression. Can be selected by providing a string t cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation. .. code-block:: python - schedule_interval="0 1 * * 3" # At 01:00 on Wednesday. + + @dag( + schedule_interval="0 1 * * 3" # At 01:00 on Wednesday. + ... + ) DeltaDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^^ Schedules data intervals with a time delta. Can be selected by providing a - :class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. + `:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. .. code-block:: python - schedule_interval=datetime.timedelta(minutes=30) - + @dag( + schedule_interval=datetime.timedelta(minutes=30) + ... + ) EventsTimetable ^^^^^^^^^^^^^^^ -Simply pass a list of ``datetime``s for the DAG to run after. Useful for timing based on sporting +Simply pass a list of ``datetimes`` for the DAG to run after. Useful for timing based on sporting events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable. The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally, @@ -89,18 +95,11 @@ first) event for the data interval, otherwise manual runs will run with a ``data .. code-block:: python - from airflow.timetables.events import EventsTimetable - @dag( - timetable=EventsTimetable( - event_dates=[ - pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), - pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), - pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago"), - ], - description="My Team's Baseball Games", - restrict_to_events=False, - ), - ..., + timetable=EventsTimetable(event_dates=[pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), + pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), + pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago")], + description="My Team's Baseball Games", + restrict_to_events=False) + ... ) - From 29f0395a92320c656d0290a63fa0faeed02694a5 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 17:58:28 -0500 Subject: [PATCH 13/18] Apply formatting suggestions from code review Co-authored-by: Tzu-ping Chung --- docs/apache-airflow/concepts/timetable.rst | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 245611a576fe1..cccc3f581b749 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -84,7 +84,7 @@ Schedules data intervals with a time delta. Can be selected by providing a EventsTimetable ^^^^^^^^^^^^^^^ -Simply pass a list of ``datetimes`` for the DAG to run after. Useful for timing based on sporting +Simply pass a list of ``datetime``s for the DAG to run after. Useful for timing based on sporting events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable. The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally, @@ -96,10 +96,14 @@ first) event for the data interval, otherwise manual runs will run with a ``data .. code-block:: python @dag( - timetable=EventsTimetable(event_dates=[pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), - pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), - pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago")], - description="My Team's Baseball Games", - restrict_to_events=False) - ... + timetable=EventsTimetable( + event_dates=[ + pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), + pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), + pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago"), + ], + description="My Team's Baseball Games", + restrict_to_events=False, + ), + ..., ) From 29b7214d75fe8557d01ffd4fba978ceae4b042d6 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Tue, 19 Apr 2022 18:03:10 -0500 Subject: [PATCH 14/18] Update docs/apache-airflow/concepts/timetable.rst Co-authored-by: Tzu-ping Chung --- docs/apache-airflow/concepts/timetable.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index cccc3f581b749..4e5e1378df865 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -72,7 +72,7 @@ DeltaDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^^ Schedules data intervals with a time delta. Can be selected by providing a - `:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. + :class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. .. code-block:: python From 63f9c49670d69d085cc7aeac2af034d3153810a0 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Wed, 20 Apr 2022 09:30:23 -0500 Subject: [PATCH 15/18] Show import of EventsTimetable on documentation --- docs/apache-airflow/concepts/timetable.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 4e5e1378df865..107a72703a654 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -95,6 +95,8 @@ first) event for the data interval, otherwise manual runs will run with a ``data .. code-block:: python + from airflow.timetables.events import EventsTimetable + @dag( timetable=EventsTimetable( event_dates=[ From 3ad8a9363565bd9e5facc059704000afc8d44de7 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Wed, 29 Jun 2022 13:21:11 -0500 Subject: [PATCH 16/18] Fix whitespace on timetables doc Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> --- docs/apache-airflow/concepts/timetable.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index 107a72703a654..a0df05f3f614b 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -59,7 +59,7 @@ CronDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^ Set schedule based on a cron expression. Can be selected by providing a string that is a valid - cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation. +cron expression to the ``schedule_interval`` parameter of a DAG as described in the :doc:`/concepts/dags` documentation. .. code-block:: python @@ -72,7 +72,7 @@ DeltaDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^^ Schedules data intervals with a time delta. Can be selected by providing a - :class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. +:class:`datetime.timedelta` or ``dateutil.relativedelta.relativedelta`` to the ``schedule_interval`` parameter of a DAG. .. code-block:: python From 0e2be06831ad006d6aa827d9e763d52dccfe1ec4 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Wed, 29 Jun 2022 15:01:34 -0500 Subject: [PATCH 17/18] Remove ellipsis from examples in timetables doc Co-authored-by: Jed Cunningham <66968678+jedcunningham@users.noreply.github.com> --- docs/apache-airflow/concepts/timetable.rst | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index a0df05f3f614b..cf54b2f4a351c 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -65,7 +65,6 @@ cron expression to the ``schedule_interval`` parameter of a DAG as described in @dag( schedule_interval="0 1 * * 3" # At 01:00 on Wednesday. - ... ) DeltaDataIntervalTimetable @@ -78,13 +77,12 @@ Schedules data intervals with a time delta. Can be selected by providing a @dag( schedule_interval=datetime.timedelta(minutes=30) - ... ) EventsTimetable ^^^^^^^^^^^^^^^ -Simply pass a list of ``datetime``s for the DAG to run after. Useful for timing based on sporting +Simply pass a list of ``datetime``\s for the DAG to run after. Useful for timing based on sporting events, planned communication campaigns, and other schedules that are arbitrary and irregular but predictable. The list of events must be finite and of reasonable size as it must be loaded every time the DAG is parsed. Optionally, @@ -107,5 +105,4 @@ first) event for the data interval, otherwise manual runs will run with a ``data description="My Team's Baseball Games", restrict_to_events=False, ), - ..., ) From c0bd4bb25eaf8349a191cc1dc2ef3a928fd68f98 Mon Sep 17 00:00:00 2001 From: Collin McNulty Date: Thu, 30 Jun 2022 12:13:16 -0500 Subject: [PATCH 18/18] Add def statement to example timetables --- docs/apache-airflow/concepts/timetable.rst | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/docs/apache-airflow/concepts/timetable.rst b/docs/apache-airflow/concepts/timetable.rst index cf54b2f4a351c..986213df8187f 100644 --- a/docs/apache-airflow/concepts/timetable.rst +++ b/docs/apache-airflow/concepts/timetable.rst @@ -63,9 +63,9 @@ cron expression to the ``schedule_interval`` parameter of a DAG as described in .. code-block:: python - @dag( - schedule_interval="0 1 * * 3" # At 01:00 on Wednesday. - ) + @dag(schedule_interval="0 1 * * 3") # At 01:00 on Wednesday. + def example_dag(): + pass DeltaDataIntervalTimetable ^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -75,9 +75,9 @@ Schedules data intervals with a time delta. Can be selected by providing a .. code-block:: python - @dag( - schedule_interval=datetime.timedelta(minutes=30) - ) + @dag(schedule_interval=datetime.timedelta(minutes=30)) + def example_dag(): + pass EventsTimetable ^^^^^^^^^^^^^^^ @@ -95,14 +95,17 @@ first) event for the data interval, otherwise manual runs will run with a ``data from airflow.timetables.events import EventsTimetable + @dag( timetable=EventsTimetable( event_dates=[ - pendulum.datetime(2022, 4, 5, 8, 27,tz="America/Chicago"), - pendulum.datetime(2022, 4, 17, 8, 27,tz="America/Chicago"), - pendulum.datetime(2022, 4, 22, 20, 50,tz="America/Chicago"), + pendulum.datetime(2022, 4, 5, 8, 27, tz="America/Chicago"), + pendulum.datetime(2022, 4, 17, 8, 27, tz="America/Chicago"), + pendulum.datetime(2022, 4, 22, 20, 50, tz="America/Chicago"), ], description="My Team's Baseball Games", restrict_to_events=False, ), ) + def example_dag(): + pass