Skip to content

Skip the ts-sdk supervisor schema check on PRs not touching ts-sdk - #69583

Merged
jason810496 merged 4 commits into
apache:mainfrom
jason810496:ci/ts-sdk/manual-schema-bump-hook
Jul 8, 2026
Merged

Skip the ts-sdk supervisor schema check on PRs not touching ts-sdk#69583
jason810496 merged 4 commits into
apache:mainfrom
jason810496:ci/ts-sdk/manual-schema-bump-hook

Conversation

@jason810496

@jason810496 jason810496 commented Jul 8, 2026

Copy link
Copy Markdown
Member

Why

CI's static-checks job runs prek --all-files, so a task-sdk-only PR that bumps the supervisor wire schema fails check-ts-sdk-supervisor-schema over a not-yet-regenerated ts-sdk/src/generated/supervisor.ts it never touched. Regenerating the ts-sdk types is the ts-sdk follow-up PR's job, not the schema author's.

What

  • Skip check-ts-sdk-supervisor-schema via the existing selective-skip mechanism (like ktlint) when no ts-sdk/ file changed; drop schema.json from the hook's files: trigger so local commits behave the same. Drift from a schema-only merge is still caught by full-tests runs (canary) and the next ts-sdk PR.
  • Regenerate supervisor.ts: Reject invalid partition keys in the create asset events API #69453 changed the wire schema without regenerating it; this PR's full static-checks run surfaced that pre-existing drift.
  • Update the selective-checks doc and tests accordingly.

Verification

  • dev/breeze selective-checks tests: 182 passed. Hook is skipped for schema-only / unrelated changes, runs for any ts-sdk/ change.
  • prek run check-ts-sdk-supervisor-schema --all-files passes after the regeneration.

Was generative AI tooling used to co-author this PR?

CI's static-checks job runs `prek --all-files`, which runs every
default-stage hook against the whole repo tree regardless of what a
given PR actually touched. Because check-ts-sdk-supervisor-schema
lists task-sdk's schema.json as a trigger file, any task-sdk-only PR
that changes the wire schema fails static checks over an unrelated,
not-yet-regenerated ts-sdk file it never intended to touch.

Move the hook to the manual stage, matching how other on-demand
checks in this file (update-notice-year, upgrade-important-versions,
update-docker-gpg-keys) are handled: it stops gating unrelated PRs
and ts-sdk owners run it themselves (`prek run --hook-stage manual
check-ts-sdk-supervisor-schema`) when they pick up a schema change.
@jason810496 jason810496 self-assigned this Jul 8, 2026
@jason810496
jason810496 requested review from Lee-W, jscheffl and potiuk July 8, 2026 02:55
The first attempt made check-ts-sdk-supervisor-schema a manual-stage
hook, but review found that removes it from CI entirely (nothing
invokes it at the manual stage), so schema drift would merge
undetected.

Reuse the existing selective-skip mechanism already used for ktlint
and the mypy-* hooks instead: the hook keeps running in CI's default
static-checks sweep, but is now skipped when neither ts-sdk/ nor the
supervisor wire schema changed, so unrelated task-sdk-only PRs no
longer fail on it while real drift is still caught.

@guan404ming guan404ming left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@Lee-W

Lee-W commented Jul 8, 2026

Copy link
Copy Markdown
Member

hmmm the CI still fails

@jason810496

Copy link
Copy Markdown
Member Author

hmmm the CI still fails

Found the root case, let me fix it shortly.

A task-sdk PR that bumps the supervisor wire schema should not be
responsible for regenerating the ts-sdk types - that is the ts-sdk
follow-up PR's job. Trigger the hook only on ts-sdk/ changes, both in
CI (selective skip) and locally (the hook's files pattern), so schema
authors are never blocked on a file they do not own. Drift merged via
a schema-only PR is still caught by full-tests runs (canary and
structural-change PRs run every hook) and by the next ts-sdk PR.
The wire schema gained an extra field on GetAssetEventByAsset and
GetAssetEventByAssetAlias in apache#69453 without the ts-sdk types being
regenerated. This PR's own changes to dev/breeze and the prek config
force a full static-checks run, where no hook is skipped, so the
check correctly surfaced that pre-existing drift and blocked CI.
@jason810496 jason810496 changed the title Make the ts-sdk supervisor schema prek hook manual Skip the ts-sdk supervisor schema check on PRs not touching ts-sdk Jul 8, 2026
@jason810496

jason810496 commented Jul 8, 2026

Copy link
Copy Markdown
Member Author

Merging this first as ts-sdk static check pass, the remaining CI failure will be handled in #69563.

All changes made by hooks:
diff --git a/providers/apache/kafka/docs/index.rst b/providers/apache/kafka/docs/index.rst
index b3fc33f..cee81d3 100644
--- a/providers/apache/kafka/docs/index.rst
+++ b/providers/apache/kafka/docs/index.rst
@@ -149,6 +149,7 @@ Install them when installing from PyPI. For example:
 Extra                 Dependencies
 ====================  ====================================================
 ``google``            ``apache-airflow-providers-google``
+``msk``               ``aws-msk-iam-sasl-signer-python>=1.0.1``
 ``common.messaging``  ``apache-airflow-providers-common-messaging>=2.0.0``
 ====================  ====================================================

@jason810496
jason810496 merged commit c96da9b into apache:main Jul 8, 2026
144 of 145 checks passed
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-3-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-3-test Commit Link

You can attempt to backport this manually by running:

cherry_picker c96da9b v3-3-test

This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@jason810496

Copy link
Copy Markdown
Member Author

No need to backport, ts-sdk merged after the v3-3-test branch cut.

@jason810496 jason810496 moved this to Done in TS-SDK Jul 20, 2026
@jason810496 jason810496 added this to the TS SDK 1.0 Beta milestone Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants