Skip to content

fix(spark): cap aiohttp<3.14 in test deps for aioresponses compat#3444

Closed
1fanwang wants to merge 1 commit into
flyteorg:masterfrom
1fanwang:fix-ci-spark-aiohttp-pin
Closed

fix(spark): cap aiohttp<3.14 in test deps for aioresponses compat#3444
1fanwang wants to merge 1 commit into
flyteorg:masterfrom
1fanwang:fix-ci-spark-aiohttp-pin

Conversation

@1fanwang

@1fanwang 1fanwang commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Why are the changes needed?

The flytekit-spark build is red on master: aioresponses mocks a ClientResponse without the stream_writer argument that aiohttp 3.14 made required, and the plugin's test deps don't cap aiohttp (CI resolves 3.14.1).

What changes were proposed in this pull request?

Cap aiohttp<3.14 in plugins/flytekit-spark/dev-requirements.in (test-only) until aioresponses supports 3.14.

How was this patch tested?

Ran plugins/flytekit-spark/tests/test_connector.py in a fresh py3.12 venv that resolved the same deps as CI (aiohttp 3.14.1, aioresponses 0.7.8):

  • Before: test_databricks_agent fails with TypeError: ClientResponse.__init__() missing 1 required keyword-only argument: 'stream_writer' (aioresponses/core.py:172) — the exact error CI hits.
  • After: installing from the capped dev-requirements.in resolves aiohttp 3.13.5; all 14 tests pass. The InvalidSpecError seen later in the CI run was a cascade off that first failure (the test never reached mock.patch.stopall()), so it clears too.

Labels

fixed

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Signed-off-by: 1fanwang <1fannnw@gmail.com>
@1fanwang

Copy link
Copy Markdown
Contributor Author

Closing — pinning aiohttp only covers part of the spark build. It also fails on unrelated pyspark 4.x errors (test_wf / test_spark_task Py4J), so the pin can't green CI on its own.

Path forward: the plugin builds need a dependency re-pin, not per-test fixes — aiohttp<3.14 in setup.py (the -U install step re-resolves a dev-requirements.in pin away), plus a pyspark<4 cap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant