fix(adapters): resolve DuckDB extension, Arrow NULL-typing, and MSSQL row-format bugs#641
Merged
Conversation
pa.Table.from_pylist infers column types from values only, so a column that is NULL in every row collapsed to Arrow null type and lost the database column's declared type. Downstream consumers misread these columns (DuckDB materializes null as INTEGER, breaking string ops). Fall back all-null value-inferred columns to string in convert_dict_to_arrow. Fixes #631
A non-required extension whose explicit install failed and whose subsequent load also failed emitted two warnings for one unusable extension. Treat install plus load as a single setup outcome: install-fail + load-success emits no warning (install detail at DEBUG), and any terminal failure of an explicitly-installed extension emits one pool.extension.setup.failed warning carrying install_error and load_error. Name-only load-only and required=True contracts are unchanged. Fixes #636
…format mssql-python returns mssql_python.Row objects that are iterable and indexable but are not tuple subclasses, while the driver declared row_format="tuple". That mismatch tripped the debug row-format fidelity assertion during dict materialization. Convert fetched rows to real tuples in dispatch_execute so the declared contract holds, matching the tuple row shape used across the SQL Server adapter family. Fixes #630
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #641 +/- ##
=======================================
Coverage 76.82% 76.82%
=======================================
Files 473 473
Lines 66108 66123 +15
Branches 9821 9825 +4
=======================================
+ Hits 50785 50799 +14
+ Misses 12031 12029 -2
- Partials 3292 3295 +3
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes three independent adapter bugs, each with regression coverage.
#631 —
select_to_arrowall-NULL columns lose their typepa.Table.from_pylistinfers types from values only, so a column that isNULLin every row collapsed to Arrownulltype and lost the DB column's declared type. Downstream consumers misread it (DuckDB materializesnullasINTEGER, soCOALESCE(col, 'x')raised a conversion error).convert_dict_to_arrownow retypes value-inferrednullcolumns tostring, which safely holds the all-null values.Closes #631
#636 — DuckDB emits two warnings for one extension setup failure
A non-required extension whose explicit install failed and whose subsequent load also failed emitted both
pool.extension.install.failedandpool.extension.load.failedfor one unusable extension. Install + load are now treated as one setup outcome:pool.extension.setup.failedcarrying bothinstall_errorandload_errorpool.extension.setup.failedrequired=Truecontracts unchangedCloses #636
#630 — MSSQL marks rows
row_format="tuple"but driver returnsmssql_python.Rowmssql_python.Rowis iterable/indexable but not atuplesubclass, so declaringrow_format="tuple"tripped the debug row-format fidelity assertion during dict materialization.dispatch_executenow materializes fetched rows into real tuples (newmaterialize_tuple_rowshelper), matching the tuple row shape used across the SQL Server adapter family.Closes #630
Testing
make type-checkclean (mypy + pyright)COALESCEon an all-NULL column, andSQLResultmaterialization of Row-like objects underrow_format="tuple"