Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/agent-memory/arch-doc-writer/MEMORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
- Four runtime images: sandbox (5 stages), gateway (2 stages), cluster (k3s base), pki-job (Alpine)
- Two build-only images: python-wheels (Linux multi-arch), python-wheels-macos (osxcross cross-compile)
- CI image: Dockerfile.ci (Ubuntu 24.04, pre-installs docker/buildx/aws/kubectl/helm/mise/uv/sccache/socat)
- Cross-compilation: `deploy/docker/cross-build.sh` shared by sandbox + gateway Dockerfiles
- Cross-compilation: `deploy/container/cross-build.sh` shared by sandbox + gateway Dockerfiles
- Sandbox image has coding-agents stage: Claude CLI (native installer), OpenCode, Codex (npm)
- Helm chart deploys a StatefulSet (NOT Deployment), PVC 1Gi at /var/openshell
- Cluster image does NOT bundle image tarballs -- components pulled at runtime from distribution registry
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci-image.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
push:
branches: [main]
paths:
- 'deploy/docker/Dockerfile.ci'
- 'deploy/container/Dockerfile.ci'
- 'mise.toml'
- 'mise.lock'
- 'tasks/**'
Expand Down Expand Up @@ -72,7 +72,7 @@ jobs:
--cache-to "type=gha,mode=max,scope=ci-image-${{ matrix.arch }}" \
--push \
-t "$ARCH_IMAGE" \
-f deploy/docker/Dockerfile.ci \
-f deploy/container/Dockerfile.ci \
.

- name: Smoke check CI image
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docker-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ jobs:
set -euo pipefail
binary="${{ needs.resolve.outputs.binary_name }}"
download_dir="prebuilt-rust-binary"
stage="deploy/docker/.build/prebuilt-binaries/${{ matrix.arch }}"
stage="deploy/container/.build/prebuilt-binaries/${{ matrix.arch }}"
found="$(find "$download_dir" -type f -name "$binary" -print -quit)"
if [[ -z "$found" ]]; then
echo "missing downloaded artifact file: $binary" >&2
Expand All @@ -233,7 +233,7 @@ jobs:
DOCKER_BUILDER: openshell
run: |
set -euo pipefail
mise exec -- tasks/scripts/docker-build-image.sh "${{ inputs.component }}" \
mise exec -- tasks/scripts/container-build-image.sh "${{ inputs.component }}" \
--cache-from "type=gha,scope=${{ inputs.component }}-${{ matrix.arch }}" \
--cache-to "type=gha,mode=max,scope=${{ inputs.component }}-${{ matrix.arch }}"

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/driver-vm-macos.yml
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ jobs:
run: |
set -euo pipefail
docker buildx build \
--file deploy/docker/Dockerfile.driver-vm-macos \
--file deploy/container/Dockerfile.driver-vm-macos \
--build-arg OPENSHELL_CARGO_VERSION="${{ inputs['cargo-version'] }}" \
--build-arg OPENSHELL_IMAGE_TAG="${{ inputs['image-tag'] }}" \
--build-arg CARGO_TARGET_CACHE_SCOPE="${{ github.sha }}" \
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,7 @@ jobs:
run: |
set -euo pipefail
docker buildx build \
--file deploy/docker/Dockerfile.cli-macos \
--file deploy/container/Dockerfile.cli-macos \
--build-arg OPENSHELL_CARGO_VERSION="${{ needs.compute-versions.outputs.cargo_version }}" \
--build-arg OPENSHELL_IMAGE_TAG=dev \
--build-arg CARGO_TARGET_CACHE_SCOPE="${{ github.sha }}" \
Expand Down Expand Up @@ -499,7 +499,7 @@ jobs:
run: |
set -euo pipefail
docker buildx build \
--file deploy/docker/Dockerfile.gateway-macos \
--file deploy/container/Dockerfile.gateway-macos \
--build-arg OPENSHELL_CARGO_VERSION="${{ needs.compute-versions.outputs.cargo_version }}" \
--build-arg OPENSHELL_IMAGE_TAG=dev \
--build-arg CARGO_TARGET_CACHE_SCOPE="${{ github.sha }}" \
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release-tag.yml
Original file line number Diff line number Diff line change
Expand Up @@ -385,7 +385,7 @@ jobs:
run: |
set -euo pipefail
docker buildx build \
--file deploy/docker/Dockerfile.cli-macos \
--file deploy/container/Dockerfile.cli-macos \
--build-arg OPENSHELL_CARGO_VERSION="${{ needs.compute-versions.outputs.cargo_version }}" \
--build-arg OPENSHELL_IMAGE_TAG="${{ needs.compute-versions.outputs.semver }}" \
--build-arg CARGO_TARGET_CACHE_SCOPE="${{ github.sha }}" \
Expand Down Expand Up @@ -618,7 +618,7 @@ jobs:
run: |
set -euo pipefail
docker buildx build \
--file deploy/docker/Dockerfile.gateway-macos \
--file deploy/container/Dockerfile.gateway-macos \
--build-arg OPENSHELL_CARGO_VERSION="${{ needs.compute-versions.outputs.cargo_version }}" \
--build-arg OPENSHELL_IMAGE_TAG="${{ needs.compute-versions.outputs.semver }}" \
--build-arg CARGO_TARGET_CACHE_SCOPE="${{ github.sha }}" \
Expand Down
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ _build/
rootfs/

# Docker build artifacts (image tarballs, packaged helm charts)
deploy/docker/.build/
deploy/container/.build/

# SBOM generated output (JSON, CSV) — release artifacts, not committed
deploy/sbom/output/
Expand Down
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,9 +178,9 @@ ocsf_emit!(event);

- Always use `uv` for Python commands (e.g., `uv pip install`, `uv run`, `uv venv`)

## Docker
## Containers

- Always prefer `mise` commands over direct docker builds (e.g., `mise run docker:build` instead of `docker build`)
- Always prefer `mise` commands over direct container builds (e.g., `mise run build:container` instead of `docker build` or `podman build`)

## Cluster Infrastructure Changes

Expand Down
6 changes: 3 additions & 3 deletions architecture/build.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ OpenShell builds these main artifacts:
|---|---|
| Gateway binary | `crates/openshell-server` |
| CLI package and Python SDK | `python/openshell` plus Rust binaries where packaged |
| Gateway and supervisor container images | `deploy/docker/Dockerfile.images` |
| Gateway and supervisor container images | `deploy/container/Dockerfile.images` |
| Helm chart | `deploy/helm/openshell` |
| VM driver/runtime assets | `crates/openshell-driver-vm` |
| Published docs site | `docs/` rendered by Fern config in `fern/` |
Expand All @@ -22,8 +22,8 @@ Sandbox community images are built outside this repository.
## Container Builds

The Docker image pipeline stages prebuilt Rust binaries, then builds container
images from `deploy/docker/Dockerfile.images`. CI builds native artifacts on the
target architecture, stages them under `deploy/docker/.build/`, and then uses
images from `deploy/container/Dockerfile.images`. CI builds native artifacts on the
target architecture, stages them under `deploy/container/.build/`, and then uses
Buildx to publish per-architecture images and multi-architecture tags.
Gateway image builds bake the corresponding supervisor image tag into the
gateway binary so Docker sandboxes do not depend on `:latest` by default.
Expand Down
4 changes: 2 additions & 2 deletions crates/openshell-driver-podman/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ sequenceDiagram
C->>C: entrypoint: /opt/openshell/bin/openshell-sandbox
```

The `supervisor` target in `deploy/docker/Dockerfile.images` copies the
The `supervisor` target in `deploy/container/Dockerfile.images` copies the
`openshell-sandbox` binary to `/openshell-sandbox` in the supervisor image.
Mounting that image at `/opt/openshell/bin` makes the binary available as
`/opt/openshell/bin/openshell-sandbox`.
Expand Down Expand Up @@ -351,4 +351,4 @@ matter compared to cluster or rootful runtimes:
netns, proxy, and relay behavior shared by all drivers.
- Container engine abstraction: `tasks/scripts/container-engine.sh` for
build/deploy support across Docker and Podman.
- Supervisor image build: `deploy/docker/Dockerfile.images`.
- Supervisor image build: `deploy/container/Dockerfile.images`.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# wheel wrapping.
#
# Usage:
# docker buildx build -f deploy/docker/Dockerfile.cli-macos \
# docker buildx build -f deploy/container/Dockerfile.cli-macos \
# --build-arg OPENSHELL_CARGO_VERSION=0.6.0 \
# --output type=local,dest=out/ .

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
# include_bytes!().
#
# Usage:
# docker buildx build -f deploy/docker/Dockerfile.driver-vm-macos \
# docker buildx build -f deploy/container/Dockerfile.driver-vm-macos \
# --build-arg OPENSHELL_CARGO_VERSION=0.6.0 \
# --build-context vm-runtime-compressed=/path/to/compressed-dir \
# --output type=local,dest=out/ .
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
# supervisor Final supervisor image (Ubuntu base, supervisor binary)
#
# Rust binaries are built natively before the image build and staged at:
# deploy/docker/.build/prebuilt-binaries/<arch>/openshell-{gateway,sandbox}
# deploy/container/.build/prebuilt-binaries/<arch>/openshell-{gateway,sandbox}
#
# For local dev (Skaffold), pass --build-arg BUILD_FROM_SOURCE=1 to compile
# binaries inside Docker instead. BuildKit only executes the selected binary
Expand Down Expand Up @@ -59,7 +59,7 @@ FROM scratch AS gateway-binary-0
ARG TARGETARCH
# --chmod=755 preserves the executable bit through actions/upload-artifact +
# download-artifact, which strip exec perms during the roundtrip.
COPY --chmod=755 deploy/docker/.build/prebuilt-binaries/${TARGETARCH}/openshell-gateway /build/out/openshell-gateway
COPY --chmod=755 deploy/container/.build/prebuilt-binaries/${TARGETARCH}/openshell-gateway /build/out/openshell-gateway

# Source-built path (local dev, BUILD_FROM_SOURCE=1)
FROM rust-builder AS gateway-binary-1
Expand All @@ -71,7 +71,7 @@ FROM scratch AS supervisor-binary-0
ARG TARGETARCH
# --chmod=755 preserves the executable bit through actions/upload-artifact +
# download-artifact, which strip exec perms during the roundtrip.
COPY --chmod=755 deploy/docker/.build/prebuilt-binaries/${TARGETARCH}/openshell-sandbox /build/out/openshell-sandbox
COPY --chmod=755 deploy/container/.build/prebuilt-binaries/${TARGETARCH}/openshell-sandbox /build/out/openshell-sandbox

# Source-built path (local dev, BUILD_FROM_SOURCE=1)
FROM rust-builder AS supervisor-binary-1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain 1.95.0
RUN pip install --no-cache-dir maturin

COPY deploy/docker/cross-build.sh /usr/local/bin/
COPY deploy/container/cross-build.sh /usr/local/bin/

FROM base AS builder

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# Shared Rust cross-compilation helpers for multi-arch Docker builds.
#
# Source this script in Dockerfile RUN layers:
# COPY deploy/docker/cross-build.sh /usr/local/bin/
# COPY deploy/container/cross-build.sh /usr/local/bin/
# RUN . cross-build.sh && install_cross_toolchain && add_rust_target
# RUN . cross-build.sh && cargo_cross_build --release -p my-crate
#
Expand Down
8 changes: 4 additions & 4 deletions deploy/helm/openshell/skaffold.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,15 +28,15 @@ build:
--target gateway \
--tag "$IMAGE" \
--load \
--file deploy/docker/Dockerfile.images \
--file deploy/container/Dockerfile.images \
.
dependencies:
paths:
- Cargo.toml
- Cargo.lock
- crates/**
- proto/**
- deploy/docker/Dockerfile.images
- deploy/container/Dockerfile.images
- crates/openshell-server/migrations/**
- image: openshell/supervisor
context: ../../..
Expand All @@ -47,15 +47,15 @@ build:
--target supervisor \
--tag "$IMAGE" \
--load \
--file deploy/docker/Dockerfile.images \
--file deploy/container/Dockerfile.images \
.
dependencies:
paths:
- Cargo.toml
- Cargo.lock
- crates/**
- proto/**
- deploy/docker/Dockerfile.images
- deploy/container/Dockerfile.images
deploy:
helm:
releases:
Expand Down
2 changes: 1 addition & 1 deletion e2e/with-docker-gateway.sh
Original file line number Diff line number Diff line change
Expand Up @@ -368,7 +368,7 @@ else
CONTAINER_ENGINE=docker \
DOCKER_PLATFORM="linux/${DAEMON_ARCH}" \
DOCKER_OUTPUT="type=local,dest=${SUPERVISOR_OUT_DIR}" \
bash "${ROOT}/tasks/scripts/docker-build-image.sh" supervisor-output
bash "${ROOT}/tasks/scripts/container-build-image.sh" supervisor-output
fi

if [ ! -f "${SUPERVISOR_BIN}" ]; then
Expand Down
4 changes: 2 additions & 2 deletions scripts/docker-cleanup.sh → scripts/container-cleanup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Clean up stale Docker images, volumes, and build cache that are not in use
# Clean up stale container images, volumes, and build cache that are not in use
# by the currently deployed OpenShell cluster.
#
# Preserves:
Expand All @@ -13,7 +13,7 @@
# - Volumes attached to running containers
#
# Usage:
# ./scripts/docker-cleanup.sh [options]
# ./scripts/container-cleanup.sh [options]
#
# Options:
# --dry-run Show what would be removed without deleting anything
Expand Down
2 changes: 1 addition & 1 deletion tasks/ci.toml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

[build]
description = "Build the whole project"
depends = ["build:rust:workspace", "build:docker", "build:python:wheel"]
depends = ["build:rust:workspace", "build:container", "build:python:wheel"]

["build:rust"]
description = "Alias for build:rust:workspace"
Expand Down
40 changes: 40 additions & 0 deletions tasks/container.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

# Container image build tasks (engine-neutral: Docker or Podman)

["build:container"]
description = "Build all container images"
depends = [
"build:container:gateway",
"build:container:supervisor",
]
hide = true

["build:container:ci"]
description = "Build the CI container image"
run = "tasks/scripts/container-build-ci.sh"
hide = true

["build:container:gateway"]
description = "Build the gateway container image"
run = "tasks/scripts/container-build-image.sh gateway"
hide = true

["build:container:supervisor"]
description = "Build the standalone supervisor container image (Ubuntu-based, for K8s pods)"
run = "tasks/scripts/container-build-image.sh supervisor"
hide = true

["build:container:multiarch"]
description = "Build multi-arch gateway and supervisor images and push to a registry"
run = "tasks/scripts/container-publish-multiarch.sh"
hide = true

["container:cleanup"]
description = "Remove stale images, volumes, and build cache not used by the current deployments"
run = "scripts/container-cleanup.sh --force"

["container:cleanup:dry-run"]
description = "Preview what container:cleanup would remove without deleting anything"
run = "scripts/container-cleanup.sh --dry-run"
Loading
Loading