fix(db): bound reader-pool statements with per-session statement_timeout (#3651) - #3997
Open
iroiro147 wants to merge 3 commits into
Open
fix(db): bound reader-pool statements with per-session statement_timeout (#3651)#3997iroiro147 wants to merge 3 commits into
iroiro147 wants to merge 3 commits into
Conversation
…rget (block#3929) The test asserted on a shared static callsite target ("trace_context_lookup_filter_test"). tracing caches callsite interest globally per static callsite rather than per-subscriber, so when the parallel lib suite ran, another test could poison that callsite's cached interest while a different subscriber was active — bypassing the thread-local with_default filter and intermittently failing the assertion that the LevelFilter::OFF filter disables the callsite. Use a target literal unique to the test statement (suffixed with the source line via concat! since enabled! requires a compile-time literal) so no other test's callsite can poison this assertion's cached interest. Removes the shared state instead of serializing around it (block#3929, fix direction 2). Signed-off-by: iroiro147 <sarthak.singh@juspay.in>
…ys settles (block#3975) Desktop's WebSocket reconnect could appear as a no-op because tauri-plugin-websocket holds a global connection-manager mutex while awaiting send(); a stuck send from a previous dead connection starved the next plugin:websocket|connect registration indefinitely. With an unsettled connectPromise, every manual Reconnect click joined the same stuck future, matching issue block#3975's report of "Reconnect does nothing, only relaunch recovers". Race the plugin connect invoke against a hard 30s timeout in a new pure helper . On timeout it rejects and the existing connection-failure/backoff path runs, so each manual reconnect starts a fresh attempt instead of joining the stuck one. The helper is injectable so the timeout path can be unit-tested without the Tauri runtime. Test: relayConnectTimeout.test.mjs — 3 cases: rejects with the timeout when the invoke never settles; resolves with the plugin wsId when it responds; clears its timer so a fast resolve never fires the fallback. Note: the desktop npm-test suite is broken at baseline in this worktree (ERR_MODULE_NOT_FOUND: typescript in test-loader.mjs) — same on clean main. This change passes standalone node --test. Signed-off-by: iroiro147 <sarthak.singh@juspay.in>
…out (block#3651) Once a routed read acquired its reader connection, nothing bounded any subsequent statement. A replica path that goes dark mid-transaction (LB failover, security-group change, standby promotion, Aurora failover) converted a routed read into an indefinitely hung request the writer-fallback could never reclaim. No statement_timeout existed anywhere in the repo; a MIDTX_STATEMENT probe ran past a 15s cap with no bound in sight. Arm a per-session statement_timeout (READER_STATEMENT_TIMEOUT_MS = 2s) on every reader connection in connect_read_pool via after_connect + set_config, matching the writer pool's floor-guard pattern at the same site. Reader sessions are read-only, so this can never abort a committed write; it only bounds in-flight SQL work and lets the caller's fallback to the writer regain control on a blackholed replica path. Wiring test: connect_read_pool_arms_statement_timeout pins the bound value and proves lazy construction still works with after_connect armed (connect_lazy dials nothing). buzz-db lib suite: 95 pass / 0 fail. Signed-off-by: iroiro147 <sarthak.singh@juspay.in>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bounds every statement issued on the reader pool (
crates/buzz-db) with a per-sessionstatement_timeout, so a replica that goes dark mid-transaction fails the statement instead of hanging the request indefinitely (#3651).Root cause
Once a routed read acquired its reader connection, nothing bounded any subsequent statement. A replica path that goes dark mid-transaction (LB failover, security-group change, standby promotion, Aurora failover) converted a routed read into an indefinitely hung request the writer-fallback could never reclaim. No
statement_timeoutexisted anywhere in the repo; aMIDTX_STATEMENTprobe ran past a 15s cap with no bound in sight.Fix
connect_read_poolnow arms a per-sessionstatement_timeout(READER_STATEMENT_TIMEOUT_MS = 2s) on every reader connection viaafter_connect+set_config('statement_timeout', ..., false)— the sameset_configpattern already used by the writer pool's floor-guard at the same site. The value is deliberately session-scoped (the trailingfalse) so it cannot leak into any wider role config.2s is comfortably above any legitimate routed read on a read-only session yet far below a hung request; on timeout Postgres sends an error, the acquire / read path fails, and the caller's fallback to the writer regains control. Collector/timer overhead and the (already sub-second) reader acquire are unaffected — this bounds only in-flight SQL work.
Reader sessions are read-only: this can never abort a committed write, it only bounds in-flight SQL.
Test
Wiring test:
connect_read_pool_arms_statement_timeoutpins the bound value and proves lazy construction still works withafter_connectarmed (connect_lazydials nothing).buzz-dblib suite: 95 pass / 0 fail (151 ignored — pre-existing Postgres-dependent skip-set).Environment note
No live replica available here (test-loader Postgres is absent in this worktree). The new wiring test passes standalone
cargo test -p buzz-db --lib; this is the same pattern as the existingconnect_read_pool_is_lazy_and_independently_sizedtest, which also never dials the network.