Skip to content

Bigquery proxy support#67218

Open
mansfieldj3 wants to merge 23 commits into
apache:mainfrom
mansfieldj3:bigquery-proxy-support
Open

Bigquery proxy support#67218
mansfieldj3 wants to merge 23 commits into
apache:mainfrom
mansfieldj3:bigquery-proxy-support

Conversation

@mansfieldj3

@mansfieldj3 mansfieldj3 commented May 19, 2026

Copy link
Copy Markdown
Contributor

Adds support for supplying a HTTP / HTTPS proxy to the bigquery hook using any of

  1. connection form widgets
  2. As a field in the hook itself as a string i.e. BigQueryHook(http_proxy="http://proxy:8080")
  3. As part of the connection extras i.e. "http_proxy": "http://proxy:8080"

Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Sonnet 4.6 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@mansfieldj3 mansfieldj3 requested a review from shahar1 as a code owner May 19, 2026 23:22
@boring-cyborg boring-cyborg Bot added area:providers kind:documentation provider:google Google (including GCP) related issues labels May 19, 2026
@mansfieldj3

mansfieldj3 commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

As an FYI, when testing in my environment the code was reformatted by Ruff which considerably expands the number of changes, if this is an issue, please let me know and I'll revert this

@mansfieldj3 mansfieldj3 marked this pull request as draft May 19, 2026 23:28
@mansfieldj3

Copy link
Copy Markdown
Contributor Author

Should be ready for review (assuming it passes the checks)

@mansfieldj3 mansfieldj3 marked this pull request as ready for review May 20, 2026 21:50
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 24, 2026
@mansfieldj3

Copy link
Copy Markdown
Contributor Author

@shahar1 can you review this when you get the chance?

@shahar1

shahar1 commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

As an FYI, when testing in my environment the code was reformatted by Ruff which considerably expands the number of changes, if this is an issue, please let me know and I'll revert this

Could you please revert the extra formatting? It makes it hard to review, thanks!

@mansfieldj3

Copy link
Copy Markdown
Contributor Author

Figured that might be the case, I'll try and update the request to revert the formatting sometime either tomorrow or early next week

@mansfieldj3

Copy link
Copy Markdown
Contributor Author

I think I've moved over the functional changes without touching the rest of the formatting, I just need to confirm it still works in my env after this

@mansfieldj3

mansfieldj3 commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Just confirming the revised version still works as intended in my environment @shahar1

@shahar1 shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution! I have a couple of comments - there's some rework to be done, mainly to make the changes minimal and precise as much as possible.
If possible, I'd like to ask adding a Dag example + a screenshot that demonstrates that the Dag can run E2E. If you're unable to do so, please provide instructions for others to do so.

Comment thread providers/google/src/airflow/providers/google/cloud/hooks/bigquery.py Outdated
Comment thread providers/google/src/airflow/providers/google/cloud/hooks/bigquery.py Outdated
Comment thread providers/google/src/airflow/providers/google/cloud/hooks/bigquery.py Outdated
Comment thread providers/google/src/airflow/providers/google/cloud/hooks/bigquery.py Outdated
Comment thread providers/google/tests/unit/google/cloud/hooks/test_bigquery.py Outdated
@mansfieldj3

mansfieldj3 commented Jun 22, 2026

Copy link
Copy Markdown
Contributor Author

Cut down unit tests, consolidated imports, reversed changes to method that wasn't needed, and added dialect / kwargs / timeout changes + tested to confirm everything was still working on my end obvs

@mansfieldj3 mansfieldj3 requested a review from shahar1 June 23, 2026 03:31
@mansfieldj3

Copy link
Copy Markdown
Contributor Author

Need to rework the unit tests. Bugger.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers kind:documentation provider:google Google (including GCP) related issues ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Google Cloud Connection support for non-environmental proxy

3 participants