RtspGStreamerLib

A lightweight .NET library for capturing RTSP video frames using GStreamer with direct P/Invoke bindings.

Features

• Zero NuGet dependencies - Direct P/Invoke to native GStreamer libraries
• Hardware acceleration - Automatic HW decoding on ARM64 devices (Khadas VIM3, Raspberry Pi, Jetson Nano)
• H.265/HEVC support - Full codec support via GStreamer (H.264 also supported)
• Precise timestamps - Access to RTSP stream PTS (Presentation Timestamp) with nanosecond precision
• Multi-platform - Windows x64, Linux x64, Linux ARM64
• BGR output - Frame data ready for OpenCV and computer vision processing

Installation

NuGet Package

dotnet add package RtspGStreamerLib

System Requirements

This library requires GStreamer to be installed on your system.

Windows x64

1. Download GStreamer from https://gstreamer.freedesktop.org/download/
2. Install both Runtime and Development packages (MSVC 64-bit)
3. Add to your PATH: C:\Program Files\gstreamer\1.0\msvc_x86_64\bin
4. Restart your terminal/IDE

Verify installation:

gst-launch-1.0 --version

Linux x64

sudo apt update
sudo apt install -y libgstreamer1.0-0 libgstreamer1.0-dev \
                    gstreamer1.0-plugins-base gstreamer1.0-plugins-good \
                    gstreamer1.0-plugins-bad gstreamer1.0-libav \
                    gstreamer1.0-tools

Linux ARM64 (Raspberry Pi, Khadas VIM3, Jetson)

sudo apt update
sudo apt install -y libgstreamer1.0-0 libgstreamer1.0-dev \
                    gstreamer1.0-plugins-base gstreamer1.0-plugins-good \
                    gstreamer1.0-plugins-bad gstreamer1.0-libav \
                    gstreamer1.0-tools gstreamer1.0-omx

Quick Start

using RtspGStreamerLib;

// Initialize GStreamer (call once at application startup)
RtspFrameCapture.Initialize();

// Create capture instance
using var capture = new RtspFrameCapture();

// Set up frame callback
capture.OnFrameReceived += (frame) =>
{
    Console.WriteLine($"Frame: {frame.Width}x{frame.Height}, Format: {frame.Format}");
    Console.WriteLine($"Stream Timestamp: {frame.StreamTimestamp}");
    Console.WriteLine($"Data size: {frame.Data.Length} bytes");

    // Process frame.Data (BGR pixel array)
    // Your processing code here...
};

// Set up error callback
capture.OnError += (error) =>
{
    Console.WriteLine($"Error: {error}");
};

// Start capturing
// useHardwareAccel: true for ARM64 devices with HW decoder support
bool useHardwareAccel = false; // Set to true on ARM64
if (capture.Start("rtsp://user:password@192.168.1.100:554/stream", useHardwareAccel))
{
    Console.WriteLine("Capture started!");

    // Keep running...
    Console.ReadKey();

    // Stop capture
    capture.Stop();
}

API Reference

RtspFrameCapture

The main class for capturing RTSP video frames.

Methods

Method	Description
static void Initialize()	Initialize GStreamer. Call once at application startup.
bool Start(string rtspUrl, bool useHardwareAccel = false)	Start capturing frames from the RTSP stream.
void Stop()	Stop the capture.
void Dispose()	Release all resources.

Events

Event	Description
OnFrameReceived	Fired when a new frame is available. Provides a VideoFrame object.
OnError	Fired when an error occurs. Provides an error message string.

Properties

Property	Type	Description
IsRunning	bool	Indicates whether capture is currently active.

VideoFrame

Represents a captured video frame.

Property	Type	Description
Data	byte[]	Raw pixel data in BGR format (3 bytes per pixel).
Width	int	Frame width in pixels.
Height	int	Frame height in pixels.
Format	string	Pixel format (typically "BGR").
StreamTimestamp	TimeSpan	Presentation timestamp from the RTSP stream.
StreamTimestampNanoseconds	ulong	Raw PTS value in nanoseconds.
ReceivedAt	DateTime	Time when frame was received in C# (includes network latency).

Frame Data Format

Frames are delivered in BGR format (3 bytes per pixel), compatible with OpenCV and most computer vision libraries.

Memory layout: [B0, G0, R0, B1, G1, R1, B2, G2, R2, ...]
Total size: Width x Height x 3 bytes

Accessing a pixel at position (x, y):

int index = (y * frame.Width + x) * 3;
byte blue  = frame.Data[index];
byte green = frame.Data[index + 1];
byte red   = frame.Data[index + 2];

Timestamps

The library provides two types of timestamps:

StreamTimestamp (Recommended)

The PTS (Presentation Timestamp) from the RTSP stream. This is the timestamp assigned by the camera and represents the actual capture time with nanosecond precision.

// Calculate time difference between frames
double deltaSeconds = (currentFrame.StreamTimestamp - previousFrame.StreamTimestamp).TotalSeconds;

ReceivedAt

The DateTime when the frame was received in your C# code. This includes network latency, decoding time, and thread scheduling delays. Use this only for logging or debugging purposes.

Hardware Acceleration

On ARM64 devices with hardware video decoding support, enable hardware acceleration for significantly lower CPU usage:

// ARM64 with v4l2 hardware decoder
capture.Start(rtspUrl, useHardwareAccel: true);

Platform	Decoder	Notes
ARM64 (HW)	v4l2h265dec	Uses hardware decoder, ~10-15% CPU
All platforms	avdec_h265	Software decoder, ~25-30% CPU

Performance

Khadas VIM3 (ARM64 with Hardware Acceleration)

Resolution	FPS	CPU Usage	RAM	Latency
2560x1440	30	~10-15%	~60 MB	~200ms
1920x1080	30	~8-12%	~40 MB	~150ms

Windows/Linux x64 (Software Decoding)

Resolution	FPS	CPU Usage	RAM	Latency
2560x1440	30	~25-30%	~100 MB	~250ms
1920x1080	30	~15-20%	~70 MB	~200ms

Advanced Configuration

Using H.264 Streams

By default, the library is configured for H.265/HEVC streams. For H.264 streams, you'll need to modify the pipeline in RtspFrameCapture.cs:

// For H.264 streams, change the pipeline to:
string pipeline =
    $"rtspsrc location=\"{rtspUrl}\" protocols=tcp latency=200 ! " +
    "rtph264depay ! h264parse ! " +
    $"{(useHardwareAccel ? "v4l2h264dec" : "avdec_h264")} ! " +
    "videoconvert ! video/x-raw,format=BGR ! " +
    "appsink name=sink emit-signals=false max-buffers=1 drop=true";

Adjusting Latency

Modify the latency parameter in the pipeline:

• Lower latency (50-100ms): Faster response, potentially less stable
• Higher latency (500ms+): More buffering, more stable, higher delay

Limiting Frame Rate

To reduce CPU usage, add a frame rate limiter to the pipeline:

"videoconvert ! videorate ! video/x-raw,framerate=10/1,format=BGR ! "

Alternative Output Formats

// RGBA (4 bytes per pixel)
"videoconvert ! video/x-raw,format=RGBA ! "

// Grayscale (1 byte per pixel)
"videoconvert ! video/x-raw,format=GRAY8 ! "

Troubleshooting

DLL/Library Not Found

Windows:

# Verify GStreamer is in PATH
where gst-launch-1.0

Linux:

# Verify library is installed
ldconfig -p | grep gstreamer
# Should show: libgstreamer-1.0.so.0

No Frames Received

1. Test your RTSP URL directly with GStreamer:

gst-launch-1.0 rtspsrc location="rtsp://your-url" ! fakesink

2. Verify the codec matches (H.264 vs H.265)

3. Try switching protocols (TCP vs UDP):

// In pipeline: protocols=tcp or protocols=udp

High CPU on ARM64

Verify hardware decoder is available:

gst-inspect-1.0 v4l2h265dec

Ensure you're enabling hardware acceleration:

capture.Start(rtspUrl, useHardwareAccel: true);

Thread Safety

• Frame callbacks execute on a background thread, not the main thread
• Use proper synchronization when accessing shared data from callbacks
• Frame data is copied to managed memory and safe to use after the callback returns
• Always call Dispose() or use using statement to release GStreamer resources

Project Structure

RtspGStreamerLib/
├── RtspGStreamerLib/           # Main library (NuGet package)
│   ├── GStreamerNative.cs      # P/Invoke bindings to GStreamer
│   ├── RtspFrameCapture.cs     # Main capture class and VideoFrame
│   └── RtspGStreamerLib.csproj
│
├── RtspGStreamerExample/       # Usage example with library reference
│   ├── Program.cs              # Basic example
│   ├── ProgramWithImage.cs     # Example with SkiaSharp image saving
│   └── ImageHelper.cs          # Image conversion utilities
│
├── RtspGStreamerInterop/       # Standalone example (no library reference)
│   ├── GStreamerNative.cs
│   ├── RtspFrameCapture.cs
│   └── Program.cs
│
└── docs/                       # Documentation
    ├── USAGE_GUIDE.md          # Complete usage guide
    ├── TIMESTAMPS.md           # Understanding RTSP timestamps
    └── COMPARISON.md           # Library comparison

Building from Source

# Clone the repository
git clone https://github.com/clovisjr/RtspGStreamerLib.git
cd RtspGStreamerLib

# Build the solution
dotnet build RtspGStreamerLib.slnx

# Build release and create NuGet package
cd RtspGStreamerLib
dotnet pack -c Release

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE.txt file for details.

Acknowledgments

• GStreamer - The multimedia framework that powers this library