feat: stream initial package installations for WASM

[dmadisetti]

📝 Summary

Allow parallel installations in pyodide

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
marimo-docs	Ready	Preview, Comment	May 27, 2026 10:33pm

[cubic-dev-ai]

4 issues found across 8 files

Architecture diagram

sequenceDiagram
    participant UI as Client UI
    participant useInstall as useInstallPackage hook
    participant WS as WebSocket
    participant Kernel as Kernel Runtime
    participant PkgCallbacks as PackagesCallbacks
    participant PkgMgr as PackageManager
    participant MicropipStream as MicropipStreamInstall
    participant DB as Database
    participant Ntf as Broadcast Notification

    Note over UI,Ntf: WASM Package Installation Flow

    UI->>useInstall: installPackages(["numpy", "pandas", "scipy"])
    useInstall->>useInstall: join packages with space separator
    useInstall->>WS: addPackage({ package: "numpy pandas scipy" })
    
    WS->>Kernel: enqueue InstallPackagesCommand
    Kernel->>PkgCallbacks: _handle_install()
    PkgCallbacks->>PkgCallbacks: _notebook_index_urls() - read PEP 723 config
    
    Note over PkgCallbacks: Mark all packages as "installing" up-front
    PkgCallbacks->>Ntf: broadcast InstallingPackageAlertNotification (installing)
    
    PkgCallbacks->>PkgMgr: stream_install(["numpy","pandas","scipy"], versions, index_urls)
    
    alt Micropip available (Pyodide)
        PkgMgr->>MicropipStream: stream_transaction_install()
        Note over MicropipStream: Resolve all requirements via micropip Transaction
        MicropipStream->>MicropipStream: gather_requirements(flat_requirements)
        
        alt Resolution failures
            MicropipStream-->>PkgMgr: yield (package, False) per failed
        end
        
        Note over MicropipStream: Install all wheels in parallel via asyncio.as_completed
        par Each wheel completes
            MicropipStream->>MicropipStream: wheel.install()
            MicropipStream-->>PkgMgr: yield (package, True/False) as completed
        end
        
        Note over MicropipStream: Handle pyodide-native packages via loadPackage
        MicropipStream->>MicropipStream: loadPackage()
        MicropipStream-->>PkgMgr: yield (package, True) per pyodide package
        
        MicropipStream->>MicropipStream: Check remaining packages via importlib.metadata
        MicropipStream-->>PkgMgr: yield (package, True/False)
        
    else Micropip unavailable or Transaction API shifted
        Note over PkgMgr: Fallback to sequential base install
        loop Each package sequentially
            PkgMgr->>PkgMgr: install(package, version)
            PkgMgr-->>PkgMgr: yield (package, True/False)
        end
    end
    
    PkgMgr-->>PkgCallbacks: async iterator of (package, success)
    loop Each result from stream
        PkgCallbacks->>PkgCallbacks: Mark package as installed/failed
        alt Success
            PkgCallbacks->>Ntf: broadcast InstallingPackageAlertNotification (installed, log "done")
        else Failure
            PkgCallbacks->>DB: add to excluded_modules
            PkgCallbacks->>Ntf: broadcast InstallingPackageAlertNotification (failed, log "done")
        end
    end
    
    PkgCallbacks->>Ntf: broadcast CompletedRunNotification
    Ntf-->>WS: notification to client
    WS-->>useInstall: response.success = True
    useInstall->>UI: showAddPackageToast for each package
    useInstall->>UI: onSuccess callback

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>