Skip to content

DuckDB: collapse best-effort install and load failures into one setup outcome #636

Description

@cofin

Summary

The DuckDB extension lifecycle added in #530 correctly separates one-time install from per-connection load. One best-effort setup failure can still emit two warnings:

  • pool.extension.install.failed
  • pool.extension.load.failed

This happens when an explicit install fails and the following load also fails because no cached extension is available. Both events describe one unusable extension on one connection.

Current behavior

DuckDBConnectionPool._create_connection() calls _install_extension_once() for explicit installs. A non-required install failure logs a warning and returns. The caller then correctly attempts load_extension(), which logs a second warning if the extension is unavailable.

Minimal network-free reproduction against current main:

import logging

from sqlspec.adapters.duckdb import pool as pool_module
from sqlspec.adapters.duckdb.pool import DuckDBConnectionPool


class BrokenConnection:
    def install_extension(self, extension, **kwargs):
        raise RuntimeError("repository unavailable")

    def load_extension(self, extension):
        raise RuntimeError("extension not installed")

    def execute(self, sql, parameters=None):
        return self

    def close(self):
        pass


pool_module.duckdb.connect = lambda **kwargs: BrokenConnection()
logging.basicConfig(level=logging.WARNING)
DuckDBConnectionPool(
    {"database": ":memory:"},
    extensions=[{"name": "encodings", "install": True}],
)._create_connection()

Current output:

WARNING:sqlspec.pool:pool.extension.install.failed
WARNING:sqlspec.pool:pool.extension.load.failed

Why load must still run

Skipping load after an install failure is not generally correct. The requested repository can be unavailable while a compatible extension is already cached locally. In that case load can succeed and the connection is usable.

Proposed behavior

For non-required explicit installs, treat install plus load as one setup outcome:

  1. Install fails, load succeeds: no warning because the extension is usable. Optionally emit the install detail at DEBUG.
  2. Install fails, load fails: emit exactly one pool.extension.setup.failed warning containing separate install_error and load_error context.
  3. Install succeeds, load fails: emit exactly one setup/load warning.
  4. Name-only load-only behavior remains unchanged.
  5. Failed installs remain uncached so a later physical connection can retry.
  6. Preserve the existing strict behavior for required=True unless its documented contract is deliberately revised separately.

One implementation shape is for _install_extension_once() to return a structured failure/status instead of logging the best-effort warning immediately. _create_connection() can then attempt load and report the final outcome.

Regression coverage

Add call-count and log-count cases to tests/unit/adapters/test_duckdb/test_extension_lifecycle.py:

  • install failure plus load failure calls both phases but records one WARNING with both errors;
  • install failure plus cached load success records no WARNING;
  • install success plus load failure records one WARNING;
  • name-only and required=True contracts remain unchanged.

Context

Observed from SQLSpec 0.54.3 while configuring prewarmed, runtime-load-only DuckDB extensions for an offline/containerized deployment. Application-side prewarming and load-only configuration remain the right production policy; this issue is about SQLSpec reporting one actionable lifecycle result rather than two warnings for one failure.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions