Video import, performance overlay, interpolation enhancements, and diagnostics

[richiemcilroy]

• Add video import feature to desktop app with progress tracking
  • Add performance overlay with FPS stats for debugging/monitoring
  • Enhance cursor and zoom interpolation with screen motion blur support
  • Improve hardware diagnostics collection for log uploads
  • Center windows on macOS with proper error handling
  • Ensure even dimensions for screen capture sizes
  • Improve video metadata extraction and decoder health monitoring
  • Refine spring solver critical damping threshold
  • Add long video export integration tests

---

Note

Introduces end‑to‑end video import and major quality/perf improvements across desktop, rendering, and export.

• New video import: start_video_import/check_import_ready commands, VideoImportProgress event, import.rs (FFmpeg transcode to H264+Opus, cancellation), tray “Import Video…”, and ImportProgress.tsx
• Rendering/interpolation: integrate ZoomFocusInterpolator, cursor spring smoothing, screen movement spring, timeline duration usage; applied in export, previews, editor, and screenshot editor
• Export: default MP4 compression now Maximum; support custom_bpp (advanced slider/UI) and wire through API; estimates updated; added long‑video export tests
• Diagnostics/logging: expanded hardware/display/camera/mic/storage collection; Windows/macOS system info and rendering status; switched to sysinfo; enhanced log payload
• Performance: new PerformanceOverlay.tsx with FPS/frame‑time/jitter; socket exposes FPS stats accessor
• Media/codec fixes: AAC resampling when >48kHz (AVFoundation); Windows audio input latency estimation; enforce even dimensions
• Desktop UX: safer macOS window centering, new editor import flow and retry; minor command/event plumbing and dependency updates

^{Written by Cursor Bugbot for commit 5e5d689. This will update automatically on new commits. Configure here.}

Greptile Overview

Greptile Summary

Overview

This PR introduces video import functionality, performance monitoring tools, and significant enhancements to the rendering pipeline's cursor and zoom interpolation systems. The changes span 39 files with 4,005 additions and 403 deletions.

Key Changes

1. Video Import Feature (New)

• New module: apps/desktop/src-tauri/src/import.rs (678 lines) implements full video import pipeline
• Transcodes external videos to Cap's project format (H264 video + Opus audio)
• Supports progress tracking via Tauri events with cancellation detection
• Validates video decodability before import
• Creates proper project structure with metadata in .cap format
• UI component ImportProgress.tsx provides user-facing progress visualization

2. Rendering Pipeline Improvements

Spring Solver Overhaul (spring_mass_damper.rs):

• Replaced iterative Euler integration with analytical spring-mass-damper solution
• Supports all damping regimes: underdamped, critically damped (within ε=0.01), and overdamped
• Provides exact solutions avoiding accumulation errors from time-stepping
• Adds rest state detection to snap to target when velocity/displacement fall below thresholds

Cursor Interpolation Enhancements (cursor_interpolation.rs):

• Added filter_cursor_shake() to remove small back-and-forth movements (< 0.015 UV in < 100ms)
• Configurable click spring parameters for snappy cursor response during clicks
• Clamped cursor positions to valid [0.0, 1.0] bounds to prevent out-of-viewport rendering
• Blended spring velocity (70%) with delta motion (30%) for smoother motion blur

Zoom & Viewport (zoom.rs):

• New calculate_zoom_and_center_for_cursor() dynamically adjusts zoom to keep cursor visible
• Reduces effective zoom level if cursor would be off-screen in auto-zoom mode
• Added spring-based easing function for smooth zoom transitions
• Fixed viewport constraint calculations for edge cases

Cursor Motion Blur & Rotation (cursor.wgsl, cursor.rs):

• Velocity-based cursor rotation (up to ±0.25 radians) for dynamic feel
• Improved motion blur using Gaussian weighting (24 samples) with asymmetric sampling
• Premultiplied alpha handling to prevent artifacts
• Blur only triggers when motion exceeds 0.5 pixels

3. Decoder Health Monitoring (avassetreader.rs)

• Tracks consecutive empty iterations (max 5) and errors (max 10)
• Automatically recreates unhealthy decoders after stalls
• Increased fallback frame search distance (30→90 frames, 60→180 at EOF)
• Dynamic cache sizing based on scrub detection vs playback
• Pool size scales with video duration (5 decoders for <10min, 8 for >30min)
• Reposition thresholds also scale (5s→10s for longer videos)

4. Diagnostics & Logging

• Comprehensive hardware info collection: CPU, memory, architecture
• Display diagnostics: resolution, refresh rate, scale factor
• Camera formats with resolution/FPS details
• Microphone sample rates and channel configuration
• Storage space tracking for recordings directory
• macOS version detection with Apple Silicon identification
• GPU name and Metal support detection

5. Performance Overlay (New)

• Real-time FPS, frame time, jitter, and dropped frame tracking
• 1-second rolling window for statistics
• Color-coded indicators (green ≥55 FPS, red <15 FPS)
• Click to copy stats to clipboard

6. Quality & Safety

• macOS window centering now catches errors instead of panicking
• Removed bash.exe.stackdump file (Windows crash dump that shouldn't be tracked)
• Screen capture dimensions ensured to be even numbers for H264 encoding compatibility

Architecture Impact

The analytical spring solver fundamentally changes how smoothing works throughout the codebase - cursor, zoom focus, and future spring-based animations will benefit from precise, time-step-independent motion. The decoder pool scaling addresses performance issues users experienced with long-form content (30+ minute recordings).

Integration Points

• Import feature connects: Tauri IPC → FFmpeg → File System → Project metadata
• Decoder health monitoring feeds into: Video playback → Cache management → Frame delivery
• Diagnostics flow: System APIs → Diagnostics collector → Log upload → Support debugging

Confidence Score: 4/5

• Safe to merge with one moderate priority logic issue in spring solver fallback path
• The PR is well-structured with comprehensive features. The video import implementation is robust with proper error handling and cancellation support. The spring solver improvement is mathematically sound in the main paths (underdamped, critically damped, overdamped with distinct roots), but has a minor edge case issue when overdamped roots are nearly equal. The decoder health monitoring, diagnostics improvements, and cursor/zoom enhancements all appear correct. The one P2 issue found is in an uncommon edge case that won't affect typical usage but should be fixed for completeness.
• crates/rendering/src/spring_mass_damper.rs - overdamped fallback logic could create discontinuities in rare edge cases

Important Files Changed

File Analysis

Filename	Score	Overview
apps/desktop/src-tauri/src/import.rs	4/5	New video import feature - converts external videos to Cap project format with H264/Opus encoding, progress tracking, and cancellation support
crates/rendering/src/spring_mass_damper.rs	5/5	Replaced iterative Euler integration with analytical spring solver supporting underdamped, overdamped, and critically damped regimes
crates/rendering/src/cursor_interpolation.rs	4/5	Added cursor shake filtering, configurable click spring, and clamped cursor positions to valid bounds (0.0-1.0)
crates/rendering/src/zoom.rs	4/5	Enhanced zoom viewport constraints to keep cursor visible, added spring-based easing, and fixed edge case handling for auto-zoom mode
crates/rendering/src/decoder/avassetreader.rs	4/5	Added decoder health monitoring with automatic recreation on stalls, increased fallback distances, and improved cache management for scrubbing
crates/rendering/src/decoder/multi_position.rs	5/5	Increased decoder pool size (5-8) and reposition thresholds (5-10s) based on video duration, improved scrub detection logic
crates/rendering/src/shaders/cursor.wgsl	4/5	Added velocity-based cursor rotation and improved motion blur with Gaussian weighting, asymmetric sampling, and premultiplied alpha handling
crates/rendering/src/layers/cursor.rs	5/5	Blended spring velocity with delta motion for smoother cursor motion blur, added rotation_params uniform for shader
crates/recording/src/diagnostics.rs	5/5	Expanded diagnostics with hardware info, display details, camera formats, microphone specs, storage info, GPU detection, and macOS version parsing
apps/desktop/src-tauri/src/logging.rs	5/5	Enhanced log upload diagnostics with comprehensive hardware, display, camera, microphone, storage, and app state information
apps/desktop/src-tauri/src/windows.rs	5/5	Added macOS-specific error handling for window centering to prevent crashes when centering fails
apps/desktop/src/routes/editor/PerformanceOverlay.tsx	5/5	New performance monitoring overlay showing real-time FPS, frame times, jitter, and dropped frames with clickable stats copying
apps/desktop/src/routes/editor/ImportProgress.tsx	5/5	New import progress UI component with animated progress indicator and fun messages, handles completion and error states
crates/export/tests/long_video_export.rs	5/5	New integration test for long video export scenarios
apps/desktop/src-tauri/src/lib.rs	5/5	Added import commands, VideoImportProgress event registration, and get_recording_meta_by_path command for external project access

Sequence Diagram

sequenceDiagram
    participant User
    participant Frontend as Frontend (ImportProgress.tsx)
    participant Tauri as Tauri Command Layer
    participant Import as Import Module (import.rs)
    participant FFmpeg as FFmpeg Decoder/Encoder
    participant FileSystem as File System
    participant Events as Event System

    User->>Frontend: Select video file to import
    Frontend->>Tauri: start_video_import(source_path)
    
    Tauri->>Import: Process import request
    Import->>Import: generate_project_name()
    Import->>Import: sanitize_filename()
    Import->>FileSystem: Create project directory (.cap)
    
    Import->>Events: emit_progress(Probing, 0.0)
    Events-->>Frontend: VideoImportProgress event
    
    Import->>FFmpeg: probe_video_can_decode()
    FFmpeg-->>Import: Decoder compatibility result
    
    alt Video not supported
        Import->>Events: emit_progress(Failed)
        Events-->>Frontend: Show error message
    else Video supported
        Import->>Import: Create initial recording metadata
        Import->>FileSystem: Save recording-meta.json
        
        Import->>Events: emit_progress(Converting, 0.0)
        Events-->>Frontend: Update progress UI
        
        Import->>FFmpeg: Open input video
        Import->>FFmpeg: Get video stream info
        Import->>FFmpeg: Get audio stream info (optional)
        Import->>FFmpeg: Create H264 video encoder
        Import->>FFmpeg: Create Opus audio encoder (if audio present)
        
        loop For each video/audio packet
            Import->>FFmpeg: Decode packet
            FFmpeg-->>Import: Decoded frame
            
            alt Frame needs scaling/conversion
                Import->>FFmpeg: Scale to even dimensions
                Import->>FFmpeg: Convert to YUV420P
            end
            
            Import->>FFmpeg: Encode frame
            FFmpeg-->>Import: Encoded packet
            
            Import->>FileSystem: check_project_exists() (cancellation check)
            
            alt Progress threshold reached (1%)
                Import->>Events: emit_progress(Converting, progress%)
                Events-->>Frontend: Update progress bar
            end
        end
        
        Import->>FFmpeg: Flush video encoder
        Import->>FFmpeg: Flush audio encoder
        Import->>FFmpeg: Write output trailers
        Import->>FileSystem: Sync files to disk
        
        Import->>Events: emit_progress(Finalizing, 0.95)
        Events-->>Frontend: Show finalizing status
        
        Import->>Import: Create final recording metadata
        Import->>FileSystem: Save complete recording-meta.json
        
        Import->>Events: emit_progress(Complete, 1.0)
        Events-->>Frontend: Show success, open editor
        
        Frontend->>User: Video ready to edit
    end

Loading

[greptile-apps]

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[cursor]

This is the final PR Bugbot will review for you during this billing cycle

Your free Bugbot reviews will reset on February 8

Details

You are on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.