Skip to content

Default the Anthropic batch model from the connection - #69624

Merged
kaxil merged 1 commit into
apache:mainfrom
astronomer:anthropic-batch-default-model
Jul 8, 2026
Merged

Default the Anthropic batch model from the connection#69624
kaxil merged 1 commit into
apache:mainfrom
astronomer:anthropic-batch-default-model

Conversation

@kaxil

@kaxil kaxil commented Jul 8, 2026

Copy link
Copy Markdown
Member

AnthropicBatchOperator made you repeat params["model"] in every request of
requests=[{"custom_id", "params"}], even though the Anthropic connection already carries a
default_model (extra['model']) that the hook's create_message / count_tokens use.
Batches were the one path that ignored it.

Now a request that omits model inherits one. Resolution order:

  1. the request's own params["model"] (unchanged; a batch can still mix models)
  2. the operator's new model argument
  3. the connection's default_model (extra['model'], else the provider default)

Design notes

  • This aligns the batch path with the rest of the hook: create_message / count_tokens
    already resolve model or self.default_model. The connection is the natural home for "which
    model", and batches now honor it.
  • params stays a verbatim passthrough of the Anthropic messages.create payload.
    create_batch fills a missing model via _apply_default_model
    without mutating the caller's request dicts, and leaves an explicit per-request model
    untouched, so mixed-model batches keep working.
  • Defaulting runs after the first-party guard and deliberately skips the Bedrock-prefix
    _resolve_model step: the Message Batches API is first-party + AWS only, so the Bedrock
    branch is unreachable here.

Usage

# Set the model once for the whole batch instead of on every request:
AnthropicBatchOperator(
    task_id="summarize",
    model="claude-opus-4-8",  # or omit and set extra['model'] on the connection
    requests=[
        {"custom_id": "t1", "params": {"max_tokens": 1024, "messages": [...]}},
        {"custom_id": "t2", "params": {"max_tokens": 1024, "messages": [...]}},
    ],
)

A request that sets its own model overrides both, so you can still mix models within one batch.

Batch requests that omit params['model'] now inherit it: from a new
AnthropicBatchOperator `model` argument, else the connection's
default_model (extra['model']). An explicit per-request model still
wins, so a batch can mix models. This aligns the batch path with the
hook's create_message/count_tokens, which already default the model
from the connection. The hook fills a missing model without mutating
the caller's request dicts.
@kaxil
kaxil merged commit 0e4b660 into apache:main Jul 8, 2026
79 checks passed
@kaxil
kaxil deleted the anthropic-batch-default-model branch July 8, 2026 18:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants