Skip to content

TypeScript SDK: embed source and integrity sections in packed bundles #69905

Description

@jason810496

Background

#69295 adds airflow-ts-pack, which embeds the bundle manifest as a single first-line comment in bundle.mjs (//# airflowMetadata=<base64 YAML>); the Node coordinator reads only that head line (bounded readline(), 1 MiB cap).

The Go SDK's AFBNDL01 executable bundle format provides two things the TS format does not (see the Trailer Layout section of task-sdk/docs/executable-bundle-spec.rst):

  • Integrity validation — a SHA-256 digest over the code region, verified by the scanner before the bundle is used.
  • An embedded source region — the original entrypoint (.ts / .js), so the UI can display the code; needed for the native TypeScript Dag feature (TypeScript SDK: native TypeScript Dag declaration #69288).

Review on #69295 agreed to keep that PR to the packing flow and refine the packing spec in a follow-up. Since bundle.mjs must remain directly runnable by node, the structure cannot be a binary EOF trailer like AFBNDL01; instead the head comment can carry the layout — a format version, start/end offsets for each section, and the digest.

What needs to happen

  1. Specify a versioned head-comment layout for .mjs bundles recording section offsets (bundled code, embedded source, metadata) and a SHA-256 digest of the code region, documented alongside AFBNDL01 in task-sdk/docs/executable-bundle-spec.rst.
  2. Update airflow-ts-pack (ts-sdk/src/cli/pack.ts) to emit the new layout and embed the original entrypoint source.
  3. Update the Node coordinator (task-sdk/src/airflow/sdk/coordinators/node/coordinator.py) to verify the digest during scan — log and skip on mismatch, with the same (path, inode, mtime, size) result caching as the AFBNDL01 reader algorithm — and expose the embedded source.

Acceptance criteria

  • A packed bundle.mjs remains directly runnable with node bundle.mjs.
  • The coordinator logs and skips a truncated or modified bundle on digest mismatch.
  • The embedded entrypoint source is retrievable for display.
  • node-bundle-spec.rst documents the TS / JS layout with its own format version.

Context

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions