feat(parallel): allow dynamic batch sizing by byte limits

[ad-claw000]

Closes #328

This PR introduces an optional max_bytes_per_batch parameter to ParallelQuery.query(). When set, the worker overrides its static batchsize strategy and instead aggregates tuples iteratively up to the specified byte threshold (estimating size mostly via the blob list byte lengths).

This allows queries involving very large blobs (like images) to batch gracefully without exceeding the DB's internal 2048 MB limits, while letting descriptor batches stay safely massive.

[ad-claw000]

Fixed the pre-commit CI failure by applying autopep8 formatting to the new log messages.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds support for dynamically sized batches in ParallelQuery based on an optional per-batch byte limit to avoid oversized requests (e.g., for large blobs).

Changes:

• Introduces max_bytes_per_batch parameter to ParallelQuery.query().
• Updates worker() to optionally build batches by estimated blob byte size instead of fixed batchsize.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ad-claw000]

Applied autopep8 formatting to the log messages to fix the pre-commit CI failure.

[Copilot]

Pull request overview

Copilot reviewed 16 out of 18 changed files in this pull request and generated 5 comments.

[ad-claw000]

Applied autopep8 formatting to the new log messages to fix the pre-commit CI failure.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[luisremis]

add more testing. for instance, test and verify that images of smaller size will produce larger batches when using AddImage.

[ad-claw000]

Added the requested test verifying that AddImage operations with smaller image blobs correctly produce larger batches when using max_bytes_per_batch, as requested in the review. See commit 9bdf19d.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[ad-claw000]

Added a unit test (test_dynamic_batching_add_image_variable_sizes) that verifies smaller images produce larger batches when using AddImage, as requested. See commit c9fcd3e.

[ad-claw000]

Also applied pre-commit (autopep8) fixes in commit 8efe264.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

[ad-claw000]

Addressed the feedback about testing AddImage with variable batch sizes. Unit tests were added in commit c9fcd3e. PTAL!