Skip to content

Document apply_function dot notation requirement for Kafka message queues#68414

Merged
shahar1 merged 2 commits into
apache:mainfrom
aeroyorch:docs-kafka-apply-function
Jul 22, 2026
Merged

Document apply_function dot notation requirement for Kafka message queues#68414
shahar1 merged 2 commits into
apache:mainfrom
aeroyorch:docs-kafka-apply-function

Conversation

@aeroyorch

@aeroyorch aeroyorch commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Add some docs about apply_function dot notation requirement for Kafka message queues.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Claude Opus 4.8 for inspecting existing docs.


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

Note

✅ Ready for review · @aeroyorch@potiuk · 2026-06-12 13:30 UTC

Thanks @aeroyorch — all checks are green and this PR is marked ready for maintainer review. The ball is with the maintainers now; a maintainer will take the next look.

Automated triage — may be imperfect.

@TimurRakhmatullin86 TimurRakhmatullin86 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified against current main (queues/kafka.py, triggers/await_message.py) — the required-for-queue / optional-for-sensor split, the Nonemessage.value().decode("utf-8") fallback, and message-as-last-positional-arg all check out.

One wording nit on the return contract, since that's exactly what this new section documents: the trigger gates on a truthy check (if event:), not is not None. So a function that returns a falsy-but-valid value (0, 0.0, "", [], {}, False) is treated as "no match" and polling continues, same as returning None. A predicate written straight from the example above — return val["amount"] > threshold — would silently never fire when it evaluates to False. The "returns any data" phrasing is inherited from the existing trigger docstring / sensors.rst, so it's a pre-existing ambiguity, but since this section is specifically about the return contract it's worth making precise here.

Suggested inline edit on the line "If it returns any data, the returned value is used as the payload of the TriggerEvent. Otherwise the trigger continues polling.":

If it returns a truthy value, that value is used as the payload of the TriggerEvent. If it returns a falsy value (None, 0, False, "", [], ...), the trigger keeps polling.

@aeroyorch
aeroyorch force-pushed the docs-kafka-apply-function branch from b6aa3cc to 0b3722a Compare July 7, 2026 06:13
@aeroyorch

Copy link
Copy Markdown
Contributor Author

Verified against current main (queues/kafka.py, triggers/await_message.py) — the required-for-queue / optional-for-sensor split, the Nonemessage.value().decode("utf-8") fallback, and message-as-last-positional-arg all check out.

One wording nit on the return contract, since that's exactly what this new section documents: the trigger gates on a truthy check (if event:), not is not None. So a function that returns a falsy-but-valid value (0, 0.0, "", [], {}, False) is treated as "no match" and polling continues, same as returning None. A predicate written straight from the example above — return val["amount"] > threshold — would silently never fire when it evaluates to False. The "returns any data" phrasing is inherited from the existing trigger docstring / sensors.rst, so it's a pre-existing ambiguity, but since this section is specifically about the return contract it's worth making precise here.

Suggested inline edit on the line "If it returns any data, the returned value is used as the payload of the TriggerEvent. Otherwise the trigger continues polling.":

If it returns a truthy value, that value is used as the payload of the TriggerEvent. If it returns a falsy value (None, 0, False, "", [], ...), the trigger keeps polling.

thanks, you're right. Review implemented :)

@aeroyorch
aeroyorch force-pushed the docs-kafka-apply-function branch from d17f2ee to d5a8c64 Compare July 7, 2026 15:47
@aeroyorch
aeroyorch force-pushed the docs-kafka-apply-function branch from d5a8c64 to 872a134 Compare July 8, 2026 17:38
@shahar1
shahar1 merged commit 74c76cb into apache:main Jul 22, 2026
86 checks passed
@aeroyorch
aeroyorch deleted the docs-kafka-apply-function branch July 22, 2026 17:32
stephen-bracken pushed a commit to stephen-bracken/airflow that referenced this pull request Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants