Give an option to override the number of shards in BQ streaming insert addresses

[dannikay]

Give an option to override the default number of shards in BQ streaming inserts. fixes #20420

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
• Update CHANGES.md with noteworthy changes.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

[github-actions]

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @jrmccluskey for label python.
R: @pabloem for label io.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[codecov]

Codecov Report

Merging #25569 (3414c5a) into master (40838f7) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master   #25569      +/-   ##
==========================================
- Coverage   72.80%   72.75%   -0.05%     
==========================================
  Files         750      751       +1     
  Lines       99314    99405      +91     
==========================================
+ Hits        72308    72325      +17     
- Misses      25644    25718      +74     
  Partials     1362     1362              

Flag	Coverage Δ
python	81.93% <100.00%> (-0.09%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
sdks/python/apache_beam/io/gcp/bigquery.py	71.41% <100.00%> (+0.06%)	⬆️
sdks/python/apache_beam/io/source_test_utils.py	88.47% <0.00%> (-1.39%)	⬇️
sdks/python/apache_beam/internal/metrics/metric.py	93.00% <0.00%> (-1.00%)	⬇️
sdks/python/apache_beam/coders/row_coder.py	94.26% <0.00%> (-0.70%)	⬇️
sdks/python/apache_beam/coders/coders.py	87.04% <0.00%> (-0.44%)	⬇️
.../apache_beam/runners/direct/transform_evaluator.py	90.04% <0.00%> (-0.28%)	⬇️
sdks/python/apache_beam/coders/coder_impl.py	93.53% <0.00%> (-0.24%)	⬇️
...on/apache_beam/runners/dataflow/dataflow_runner.py	81.74% <0.00%> (-0.15%)	⬇️
...ks/python/apache_beam/coders/coders_test_common.py	98.58% <0.00%> (ø)
...n/apache_beam/ml/inference/tensorflow_inference.py	0.00% <0.00%> (ø)
... and 4 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[dannikay]

A friendly reminder for code review @jrmccluskey @pabloem

[jrmccluskey]

Run Python PreCommit

[jrmccluskey]

Is there a reason you don't have the old default value set here? It would help mitigate issues with backwards compatibility for users who aren't using the test_client and max_retries args either

[dannikay]

Done. Thanks!

[jrmccluskey]

LGTM from a Python code standpoint, waiting on @pabloem for a look over from an IO standpoint

[dannikay]

Friendly reminder for review~ @pabloem

[dannikay]

Friendly reminder for review~ @pabloem

[dannikay]

Another friendly reminder for review~ @pabloem

[jrmccluskey]

Looks like Pablo isn't going to get to it, but based on what I'm seeing and the testing I'm comfortable merging. Thank you for your contribution, and sorry for the wait!

[dannikay]

Thank you @jrmccluskey !