Skip to content
Merged

Dev #90

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
a9851a4
Testing (#9)
vsilent Mar 10, 2021
9d21449
Update README.md
vsilent Mar 10, 2021
4f22fac
Add editorconfig. Append gitignore fot emacs
diaevd Mar 12, 2021
ee745f2
editorconfig
diaevd Mar 12, 2021
f5f08cf
conflict resolved
Mar 23, 2021
4b35e9b
Update README.md
vsilent Mar 23, 2021
6e4ec6e
Update actix requirement from 0.10 to 0.11 (#18)
dependabot[bot] Mar 29, 2021
271da4d
Update actix-cors requirement from 0.3.0 to 0.5.4 (#19)
dependabot[bot] Mar 29, 2021
6b3d5a5
Update bcrypt requirement from 0.8.2 to 0.9.0 (#17)
dependabot[bot] Mar 29, 2021
5546dda
Update env_logger requirement from 0.7.1 to 0.8.3 (#16)
dependabot[bot] Mar 29, 2021
0bb944c
Update bigdecimal requirement from 0.0.14 to 0.2.0 (#15)
dependabot[bot] Mar 29, 2021
a26aa27
Update actix-service requirement from 1.0.6 to 2.0.0 (#23)
dependabot[bot] May 10, 2021
f8ad5e8
Bump codacy/codacy-analysis-cli-action from 2.0.1 to 3.0.1 (#24)
dependabot[bot] May 10, 2021
8c9fef1
Bump codacy/codacy-analysis-cli-action from 3.0.1 to 3.0.2 (#25)
dependabot[bot] May 17, 2021
25bae7f
Bump actions/cache from 2.1.4 to 2.1.5 (#26)
dependabot[bot] May 17, 2021
51f0797
Bump codacy/codacy-analysis-cli-action from 3.0.2 to 3.0.3 (#28)
dependabot[bot] Jun 16, 2021
290a8e0
Update actix requirement from 0.11 to 0.12 (#31)
dependabot[bot] Jun 16, 2021
339d5f7
Bump actions/cache from 2.1.5 to 2.1.6 (#29)
dependabot[bot] Jun 16, 2021
38871bf
Update bcrypt requirement from 0.9.0 to 0.10.0 (#32)
dependabot[bot] Jun 21, 2021
d67b5ed
Bump codacy/codacy-analysis-cli-action from 3.0.3 to 4.0.0 (#35)
dependabot[bot] Jul 26, 2021
d82371b
Update env_logger requirement from 0.8.3 to 0.9.0 (#34)
dependabot[bot] Jul 26, 2021
663d7b5
Update bigdecimal requirement from 0.2.0 to 0.3.0 (#37)
dependabot[bot] Aug 30, 2021
7c25842
Update actix-tls requirement from 2.0.0 to 3.0.0 (#39)
dependabot[bot] Dec 27, 2021
3e78c5e
Remove unused imports, list docker containers added
Feb 7, 2022
6820bd2
actix-web upgrade
Feb 9, 2022
2cdc782
shell commands
Jul 23, 2024
97b6a76
shell commands
Jul 23, 2024
2209637
rustscan, openssl binaries added
Jul 23, 2024
de2ec55
rustscan, openssl binaries added
Jul 23, 2024
728fe46
phase 1 files
Mar 14, 2026
dceae83
Broken, integrating bollard for container security check
Mar 16, 2026
bc49338
Update README with new logo and project details
vsilent Mar 16, 2026
f789f43
Revise README with new images and title case
vsilent Mar 17, 2026
f987859
diesel replaced with r2d2 and rusqlite
Mar 17, 2026
f750dce
Merge branch 'main' of https://github.com/vsilent/stackdog
Mar 17, 2026
e72c96b
ebpf files
Mar 17, 2026
8759564
refactoring, ebpf / containers
Mar 19, 2026
2f6db16
feat(cli): add clap subcommands (serve/sniff) + sniff config
Mar 30, 2026
27f3d6f
feat(sniff): log source discovery + database persistence
Mar 30, 2026
c9164c5
feat(sniff): log reader trait + File/Docker/Journald implementations
Mar 30, 2026
dc09c3e
feat(sniff): AI log analysis with OpenAI and pattern backends
Mar 30, 2026
b358eac
feat(sniff): consume mode — zstd compression, dedup, log purge
Mar 30, 2026
1658cc6
feat(sniff): reporter + orchestrator loop
Mar 30, 2026
2f38be3
feat(sniff): REST API for log sources and summaries
Mar 30, 2026
5f4b05c
docs: update CHANGELOG and README for sniff feature
Mar 30, 2026
ac3e579
Merge feature/sniff-logs: log discovery, AI analysis, archival
Mar 30, 2026
49b37aa
chore: remove task files from repo and gitignore
Mar 30, 2026
c8e5c90
feat: add curl-based binary installation
Mar 30, 2026
83cf1d2
docs: fix ML module status — stub infrastructure, not in progress
Mar 30, 2026
b2ffc0a
feat(cli): add --ai-model and --ai-api-url flags to sniff command
Mar 31, 2026
01942f8
feat(sniff): add debug logging and robust LLM JSON extraction
Mar 31, 2026
9504301
feat(alerting): implement real Slack webhook notifications
Mar 31, 2026
72d7515
Update docker.yml
vsilent Mar 31, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 25 additions & 2 deletions .env.sample
Original file line number Diff line number Diff line change
@@ -1,2 +1,25 @@
APP_HOST=localhost
APP_PORT=5000
#APP_HOST=localhost
# in container
APP_HOST=0.0.0.0

APP_PORT=5000
DATABASE_URL=stackdog.db
RUST_BACKTRACE=full

# Log Sniff Configuration
#STACKDOG_LOG_SOURCES=/var/log/syslog,/var/log/auth.log
#STACKDOG_SNIFF_INTERVAL=30
#STACKDOG_SNIFF_OUTPUT_DIR=./stackdog-logs/

# AI Provider Configuration
# Supports OpenAI, Ollama (http://localhost:11434/v1), or any OpenAI-compatible API
#STACKDOG_AI_PROVIDER=openai
#STACKDOG_AI_API_URL=http://localhost:11434/v1
#STACKDOG_AI_API_KEY=
#STACKDOG_AI_MODEL=llama3

# Notification Channels
# Slack: create an incoming webhook at https://api.slack.com/messaging/webhooks
#STACKDOG_SLACK_WEBHOOK_URL=https://hooks.slack.com/services/T.../B.../xxxxx
# Generic webhook endpoint for alert notifications
#STACKDOG_WEBHOOK_URL=https://example.com/webhook
113 changes: 113 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
# Stackdog Security — Copilot Instructions

## What This Project Is

Stackdog is a Rust-based security platform for Docker containers and Linux servers. It collects events via eBPF syscall monitoring, runs them through a rule/signature engine and optional ML anomaly detection, manages firewall responses (nftables/iptables + container quarantine), and exposes a REST + WebSocket API consumed by a React/TypeScript dashboard.

## Workspace Structure

This is a Cargo workspace with two crates:
- `.` — Main crate (`stackdog`): HTTP server, all security logic
- `ebpf/` — Separate crate (`stackdog-ebpf`): eBPF programs compiled for the kernel (uses `aya-ebpf`)

## Build, Test, and Lint Commands

```bash
# Build
cargo build
cargo build --release

# Tests
cargo test --lib # Unit tests only (in-source)
cargo test --all # All tests including integration
cargo test --lib -- events:: # Run tests for a specific module
cargo test --lib -- rules::scorer # Run a single test by name prefix

# Code quality
cargo fmt --all
cargo clippy --all
cargo audit # Dependency vulnerability scan

# Benchmarks
cargo bench

# Frontend (in web/)
npm test
npm run lint
npm run build
```

## Environment Setup

Requires a `.env` file (copy `.env.sample`). Key variables:
```
APP_HOST=0.0.0.0
APP_PORT=5000
DATABASE_URL=stackdog.db
RUST_BACKTRACE=full
```

System dependencies (Linux): `libsqlite3-dev libssl-dev clang llvm pkg-config`

## Architecture

```
Collectors (Linux only) Rule Engine Response
eBPF syscall events → Signatures → nftables/iptables
Docker daemon events → Threat scoring → Container quarantine
Network events → ML anomaly det. → Alerting

REST + WebSocket API
React/TypeScript UI
```

**Key src/ modules:**

| Module | Purpose |
|---|---|
| `events/` | Core event types: `SyscallEvent`, `SecurityEvent`, `NetworkEvent`, `ContainerEvent` |
| `rules/` | Rule engine, signature database, threat scorer |
| `alerting/` | `AlertManager`, notification channels (Slack/email/webhook) |
| `collectors/` | eBPF loader, Docker daemon events, network collector (Linux only) |
| `firewall/` | nftables management, iptables fallback, `QuarantineManager` (Linux only) |
| `ml/` | Candle-based anomaly detection (optional `ml` feature) |
| `correlator/` | Event correlation engine |
| `baselines/` | Baseline learning for anomaly detection |
| `database/` | SQLite connection pool (`r2d2` + raw `rusqlite`), repositories |
| `api/` | actix-web REST endpoints + WebSocket |
| `response/` | Automated response action pipeline |

## Key Conventions

### Platform-Gating
Linux-only modules (`collectors`, `firewall`) and deps (aya, netlink) are gated:
```rust
#[cfg(target_os = "linux")]
pub mod firewall;
```
The `ebpf` and `ml` features are opt-in and must be enabled explicitly:
```bash
cargo build --features ebpf
cargo build --features ml
```

### Error Handling
- Use `anyhow::{Result, Context}` for application/binary code
- Use `thiserror` for library error types
- Never use `.unwrap()` in production code; use `?` with `.context("...")`

### Database
The project uses raw `rusqlite` with `r2d2` connection pooling. `DbPool` is `r2d2::Pool<SqliteConnectionManager>`. Tables are created with `CREATE TABLE IF NOT EXISTS` in `database::connection::init_database`. Repositories are in `src/database/repositories/` and receive a `&DbPool`.

### API Routes
Each API sub-module exports a `configure_routes(cfg: &mut web::ServiceConfig)` function. All routes are composed in `api::configure_all_routes`, which is the single call site in `main.rs`.

### Test Location
- **Unit tests**: `#[cfg(test)] mod tests { ... }` inside source files
- **Integration tests**: `tests/` directory at workspace root

### eBPF Programs
The `ebpf/` crate is compiled separately for the Linux kernel. User-space loading is handled by `src/collectors/ebpf/` using the `aya` library. Kernel-side programs use `aya-ebpf`.

### Async Runtime
The main binary uses `#[actix_rt::main]`. Library code uses `tokio`. Avoid mixing runtimes.
2 changes: 1 addition & 1 deletion .github/workflows/codacy-analysis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ jobs:

# Execute Codacy Analysis CLI and generate a SARIF output with the security issues identified during the analysis
- name: Run Codacy Analysis CLI
uses: codacy/codacy-analysis-cli-action@2.0.1
uses: codacy/codacy-analysis-cli-action@4.0.0
with:
# Check https://github.com/codacy/codacy-analysis-cli#project-token to get your project token from your Codacy repository
# You can also omit the token and run the tools that support default configurations
Expand Down
12 changes: 7 additions & 5 deletions .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ on:
jobs:
cicd-linux-docker:
name: Cargo and npm build
runs-on: ubuntu-latest
#runs-on: ubuntu-latest
runs-on: [self-hosted, linux]
steps:
- name: Checkout sources
uses: actions/checkout@v2
Expand All @@ -26,7 +27,7 @@ jobs:
components: rustfmt, clippy

- name: Cache cargo registry
uses: actions/cache@v2.1.4
uses: actions/cache@v2.1.6
with:
path: ~/.cargo/registry
key: docker-registry-${{ hashFiles('**/Cargo.lock') }}
Expand All @@ -35,7 +36,7 @@ jobs:
docker-

- name: Cache cargo index
uses: actions/cache@v2.1.4
uses: actions/cache@v2.1.6
with:
path: ~/.cargo/git
key: docker-index-${{ hashFiles('**/Cargo.lock') }}
Expand All @@ -48,7 +49,7 @@ jobs:
head -c16 /dev/urandom > src/secret.key

- name: Cache cargo build
uses: actions/cache@v2.1.4
uses: actions/cache@v2.1.6
with:
path: target
key: docker-build-${{ hashFiles('**/Cargo.lock') }}
Expand Down Expand Up @@ -135,7 +136,8 @@ jobs:

cicd-docker:
name: CICD Docker
runs-on: ubuntu-latest
#runs-on: ubuntu-latest
runs-on: [self-hosted, linux]
needs: cicd-linux-docker
steps:
- name: Download app archive
Expand Down
77 changes: 77 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
name: Release

on:
push:
tags:
- "v*"

permissions:
contents: write

env:
CARGO_TERM_COLOR: always

jobs:
build:
name: Build ${{ matrix.target }}
runs-on: ubuntu-latest
strategy:
matrix:
include:
- target: x86_64-unknown-linux-gnu
artifact: stackdog-linux-x86_64
- target: aarch64-unknown-linux-gnu
artifact: stackdog-linux-aarch64

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@stable
with:
targets: ${{ matrix.target }}

- name: Install cross
run: cargo install cross --git https://github.com/cross-rs/cross

- name: Build release binary
run: cross build --release --target ${{ matrix.target }}

- name: Package
run: |
mkdir -p dist
cp target/${{ matrix.target }}/release/stackdog dist/stackdog
cd dist
tar czf ${{ matrix.artifact }}.tar.gz stackdog
sha256sum ${{ matrix.artifact }}.tar.gz > ${{ matrix.artifact }}.tar.gz.sha256

- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.artifact }}
path: |
dist/${{ matrix.artifact }}.tar.gz
dist/${{ matrix.artifact }}.tar.gz.sha256

release:
name: Create GitHub Release
needs: build
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Download all artifacts
uses: actions/download-artifact@v4
with:
path: artifacts
merge-multiple: true

- name: Create release
uses: softprops/action-gh-release@v2
with:
generate_release_notes: true
files: |
artifacts/*.tar.gz
artifacts/*.sha256
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,8 @@ Cargo.lock
# End of https://www.gitignore.io/api/rust,code

.idea
<<<<<<< HEAD
=======
*.db
>>>>>>> testing
docs/tasks/
Loading