Skip to content

Migrate common.ai provider to pydantic-ai 2.x and remove the <2 cap#69358

Merged
kaxil merged 1 commit into
apache:mainfrom
astronomer:migrate-common-ai-to-pydantic-ai-2x
Jul 3, 2026
Merged

Migrate common.ai provider to pydantic-ai 2.x and remove the <2 cap#69358
kaxil merged 1 commit into
apache:mainfrom
astronomer:migrate-common-ai-to-pydantic-ai-2x

Conversation

@kaxil

@kaxil kaxil commented Jul 3, 2026

Copy link
Copy Markdown
Member

Migrates apache-airflow-providers-common-ai to pydantic-ai 2.x and removes the temporary <2 cap added in #68933.

pydantic-ai 2.x removed the instrument keyword from the Agent() and Agent.from_file() constructors; it is now only the agent.instrument property. PydanticAIHook.create_agent forwarded a caller-supplied instrument straight into the constructor, so create_agent(..., instrument=...) raised TypeError: got an unexpected keyword argument 'instrument' on 2.x. The hook now pops a caller-supplied instrument out of the constructor kwargs and applies it through the property, so a caller still overrides the provider's auto-instrumentation.

Changes

  • Instrument handling (hooks/pydantic_ai.py): pop instrument before constructing the agent, then apply it via the agent.instrument property.
  • Instrumentation format v4 to v5 (observability.py): formats 2 to 4 are deprecated in 2.x and emit a warning; v5 is the current default.
  • Type annotations: Agent[None, ...] becomes Agent[object, ...] in the hook overloads and operators, matching the 2.x generic-deps default (object, not None). Type-checking only; runtime unchanged.
  • Dependency floor: pydantic-ai-slim>=1.99.0,<2 becomes >=2.0.0; re-locked (pydantic-ai-slim and pydantic-graph 1.107.0 to 2.1.0).

Behavior changes (from pydantic-ai 2.x, surfaced through the provider)

  • Telemetry: agent-run spans now report token usage under gen_ai.aggregated_usage.*, while per-model-call spans keep gen_ai.usage.*. This avoids double-counting in backends that sum a parent span and its children. Documented in observability.rst and asserted in the end-to-end span test. Dashboards or alerts reading run-level usage from gen_ai.usage.* should switch to gen_ai.aggregated_usage.*. Tracing is opt-in ([common.ai] otel_export_enabled) and off by default.
  • end_strategy default flipped from early to graceful: function tools requested in the same response as a successful output tool now run instead of being skipped.

Design notes

  • The floor is raised to >=2.0.0 rather than keeping 1.x/2.x dual support: format v5 does not exist on 1.x, dual-version branches would be untestable in CI (the lock resolves a single version), and the provider is pre-1.0. Raising a third-party floor is allowed; the release-manager-only rule applies to apache-airflow-providers-* dependencies.

Known follow-up

  • The code-mode and skills extras still carry their pre-2.x adapter floors (pydantic-ai-harness>=0.3.0, pydantic-ai-skills>=0.11.0). The 2.x-native pydantic-ai-harness (0.5.0, which requires pydantic-ai-slim>=2.1.0) currently falls outside the repository's exclude-newer window, so raising those extra floors is left as a follow-up once it resolves.

Closes #69122

pydantic-ai 2.x removed the ``instrument`` keyword from the ``Agent()`` and
``Agent.from_file()`` constructors (it is now only the ``agent.instrument``
property), which broke ``PydanticAIHook.create_agent`` when a caller passed
``instrument``. Pop a caller-supplied ``instrument`` out of the constructor
kwargs and apply it through the property instead, so the caller still wins over
the provider's auto-instrumentation.

- Bump the OTel GenAI semantic-convention format to version 5 (formats 2-4 are
  deprecated in 2.x and warn). Agent-run token usage now reports under
  ``gen_ai.aggregated_usage.*`` while model-request spans keep ``gen_ai.usage.*``;
  documented in observability.rst and asserted in the end-to-end span test.
- Update ``Agent[None, ...]`` annotations to ``Agent[object, ...]`` to match the
  2.x generic-deps default (object, not None).
- Raise the floor to ``pydantic-ai-slim>=2.0.0`` and re-lock (slim + pydantic-graph
  1.107.0 -> 2.1.0).

Closes apache#69122
@kaxil
kaxil merged commit f87aa58 into apache:main Jul 3, 2026
89 of 90 checks passed
@kaxil
kaxil deleted the migrate-common-ai-to-pydantic-ai-2x branch July 3, 2026 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migrate common.ai provider to pydantic-ai 2.x and remove the <2 cap

2 participants