Skip to content

Add openlineage system tests for missing use cases#68664

Merged
mobuchowski merged 1 commit into
apache:mainfrom
kacpermuda:ol-system-tests-extension
Jun 17, 2026
Merged

Add openlineage system tests for missing use cases#68664
mobuchowski merged 1 commit into
apache:mainfrom
kacpermuda:ol-system-tests-extension

Conversation

@kacpermuda

@kacpermuda kacpermuda commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

PR improving only OL system tests and just few more nits around typing: whitelisting "ol" and "OL" in word spelling, and extending types in openlineage/api with Dataset as addition to InputDataset and OutputDataset to prevent mypy errors for users. I need to improve typing in OL client, will followup with PR removing these type: ignore lines.

Adding some missing test cases for OL system tests:

  • custom operator with all custom OL methods
  • custom operator with some OL methods
  • all facets (custom ol operator producing all OL facets so that we can validate if nothing is broken in python client where we import them from)
  • timetables serialization for timetables that are missing now
  • edge labels
  • hitl
  • manual lineage api functions from openlineage/api
  • fail task events
  • fail dag events

Also some json formatting, improved logging in test operator, added possibility for test expected templates to cover all AF versions after some, f.e. 3.3+ etc.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@kacpermuda kacpermuda requested a review from mobuchowski as a code owner June 17, 2026 08:54
@kacpermuda kacpermuda force-pushed the ol-system-tests-extension branch 3 times, most recently from 3509e65 to 6646b0b Compare June 17, 2026 11:05
@kacpermuda kacpermuda force-pushed the ol-system-tests-extension branch from 6646b0b to 2662ac6 Compare June 17, 2026 12:28
@mobuchowski mobuchowski merged commit 674544c into apache:main Jun 17, 2026
86 checks passed
@kacpermuda kacpermuda deleted the ol-system-tests-extension branch June 17, 2026 13:40
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 17, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:

* openlineage: type-hint widening on the public emit_dataset_lineage /
  emit_query_lineage helpers (apache#68664) was not in the original wave; include it
  as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
  triggers (apache#68685) is recorded as a Misc entry in each provider's pending
  section (no version escalation - all three were already bumped higher).
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 18, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:

* openlineage: type-hint widening on the public emit_dataset_lineage /
  emit_query_lineage helpers (apache#68664) was not in the original wave; include it
  as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
  triggers (apache#68685) is recorded as a Misc entry in each provider's pending
  section (no version escalation - all three were already bumped higher).
* common-ci
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 18, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:

* openlineage: type-hint widening on the public emit_dataset_lineage /
  emit_query_lineage helpers (apache#68664) was not in the original wave; include it
  as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
  triggers (apache#68685) is recorded as a Misc entry in each provider's pending
  section (no version escalation - all three were already bumped higher).
* common-ci
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 18, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:

* openlineage: type-hint widening on the public emit_dataset_lineage /
  emit_query_lineage helpers (apache#68664) was not in the original wave; include it
  as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
  triggers (apache#68685) is recorded as a Misc entry in each provider's pending
  section (no version escalation - all three were already bumped higher).
* common-ci
shahar1 added a commit to shahar1/airflow that referenced this pull request Jun 19, 2026
Several PRs landed on main after the initial 2026-06-16 preparation and belong
in this wave:

* openlineage: type-hint widening on the public emit_dataset_lineage /
  emit_query_lineage helpers (apache#68664) was not in the original wave; include it
  as a new 2.18.1 release.
* amazon, cncf.kubernetes, google: the task_instance mypy fix in provider
  triggers (apache#68685) is recorded as a Misc entry in each provider's pending
  section (no version escalation - all three were already bumped higher).
* common-ci
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants