nothing is working

[Schiewatir]

one of the worst project ever that even the providers didn't check if everything will work or not, no kafka stream is working , no any data would store in MinIO , absolutely frustrating, also very bad documenting.

[Schiewatir]

duckdb
DuckDB v1.3.2 (Ossivalis) 0b83e5d2f6
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D LOAD airport;
D ATTACH 'boilstream' (TYPE AIRPORT, location 'grpc://localhost:50051/');
D SELECT COUNT(*) as total_sales FROM boilstream.s3.products;
IO Error:
Airport @ grpc://localhost:50051/boilstream/s3/products: Flight returned unavailable error, with message: failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:50250: Failed to connect to remote host: Connection refused.
D SELECT table_catalog, table_schema, table_name
FROM information_schema.tables
WHERE table_catalog = 'boilstream';
┌───────────────┬──────────────┬──────────────┐
│ table_catalog │ table_schema │ table_name │
│ varchar │ varchar │ varchar │
├───────────────┼──────────────┼──────────────┤
│ boilstream │ s3 │ stock_prices │
│ boilstream │ s3 │ customers │
│ boilstream │ s3 │ products │
└───────────────┴──────────────┴──────────────┘

[Schiewatir]

Key Insights from My Findings

1. Namespace Configuration Critical
   ✅ Fixed: Adding "namespace": "s3" in config.yaml solved schema discovery

   ✅ Result: s3 schema now properly appears in information_schema.tables

2. Hybrid Interface Design
   BoilStream uses two complementary interfaces:

PGWire (Port 5432) - Schema Management
✅ DDL Operations: CREATE TABLE works for simple schemas
✅ Complex Analytics: Full SQL support with JOINs, subqueries
✅ Schema Discovery: Full information_schema support
❌ Complex Types: STRUCT/ARRAY in DDL cause catalog errors

Airport/Flight RPC (Port 50051) - Data Ingestion
✅ High-Speed INSERT: Optimized for bulk data ingestion
❌ Schema Inference: Tables are not created automatically via INSERT
✅ DuckDB Integration: Full generate_series, analytics functions
❌ DDL Operations: CREATE TABLE not supported
❌ Complex Queries: Subqueries cause connection errors

3. Connection Refresh Pattern
   🔄 Discovery: Create tables in PGWire → Re-attach Airport → Tables appear in DuckDB

   🔄 Workflow: Use both interfaces for their strengths

[Schiewatir]

D LOAD airport;
D ATTACH 'boilstream' (TYPE AIRPORT, location 'grpc://localhost:50051');
D CREATE TABLE boilstream.s3.people (
name VARCHAR,
age INT,
tags VARCHAR[]
);
IO Error:
Airport @ grpc://localhost:50051/: create_table: reading result Flight returned unimplemented error, with message: Unknown action type: create_table