Skip to content

add muse2 main branch install to ci #50

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Aurashk merged 16 commits into from
May 1, 2026
Merged

add muse2 main branch install to ci #50

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
267aeec
add muse2 main branch install to ci
Aurashk Apr 24, 2026
e76effd
correct cargo install command
Aurashk Apr 24, 2026
9cfb02c
clean up CI naming
Aurashk Apr 24, 2026
baab108
add check the the file at muse2 path is executable
Aurashk Apr 27, 2026
e335c0c
remove string check
Aurashk Apr 27, 2026
ab0b170
make codecov token required instead of checking it's there
Aurashk Apr 27, 2026
e2fc632
add locked flag to cargo install
Aurashk Apr 27, 2026
351e279
on windows must be exe
Aurashk Apr 27, 2026
dd6004c
swap curl for gh cli for downloading releases
Aurashk Apr 27, 2026
5c57b11
simplify cargo install in CI, path not required in cargo
Aurashk May 1, 2026
1d456a0
bump codecov to v6
Aurashk May 1, 2026
06c8953
Merge branch 'main' into add-muse2-main-to-ci
Aurashk May 1, 2026
5c6eada
add scheduled ci run on mondays at 5
Aurashk May 1, 2026
8974828
Merge branch 'add-muse2-main-to-ci' of https://github.com/EnergySyste…
Aurashk May 1, 2026
ddec877
seperate windows verify muse2 exe logic
Aurashk May 1, 2026
d2faf70
remove unused ci input
Aurashk May 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/actions/setup-muse2-main/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
name: Setup MUSE2 from main
description: Build and install MUSE2 from the MUSE2 main branch via cargo install
runs:
using: composite
steps:
- name: Install Rust toolchain
uses: actions-rust-lang/setup-rust-toolchain@v1

- name: Build and install MUSE2 from main (cargo)
shell: bash
run: |
cargo install \
--git https://github.com/EnergySystemsModellingLab/MUSE2 \
--branch main \
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cargo install from a git repo should generally be run with --locked to ensure the build uses the repo's Cargo.lock (and fails if it's out of date) rather than implicitly updating dependencies, which can make CI flaky and less reproducible.

Suggested change
--branch main \
--branch main \
--locked \

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

@Aurashk Aurashk Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems sensible ?

--locked
Comment thread
Aurashk marked this conversation as resolved.
41 changes: 41 additions & 0 deletions .github/actions/setup-muse2-release/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
name: Setup MUSE2 from latest release
description: Download latest MUSE2 release asset for current OS and configure MUSE2_PATH
inputs:
github_token:
description: GitHub token for API access
required: true
muse2_exe:
description: MUSE2 executable name for the current OS
required: true
runs:
using: composite
steps:
- name: Download MUSE2 release asset
shell: bash
env:
GITHUB_TOKEN: ${{ inputs.github_token }}
run: |
gh release download \
--repo EnergySystemsModellingLab/MUSE2 \
--pattern "${{ runner.os == 'Windows' && 'muse2_windows.zip' || runner.os == 'macOS' && 'muse2_macos_arm.tar.gz' || 'muse2_linux.tar.gz' }}" \
--output "muse2_asset_download" \
--clobber

- name: Extract and set MUSE2_PATH (Linux / macOS)
if: runner.os == 'Linux' || runner.os == 'macOS'
shell: bash
run: |
mkdir -p muse2_bin
tar -xf muse2_asset_download -C muse2_bin
chmod +x "muse2_bin/${{ inputs.muse2_exe }}"
echo "MUSE2_PATH=${{ github.workspace }}/muse2_bin/${{ inputs.muse2_exe }}" >> "$GITHUB_ENV"

- name: Extract and set MUSE2_PATH (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
New-Item -ItemType Directory -Force -Path muse2_bin | Out-Null
Rename-Item -Path "muse2_asset_download" -NewName "muse2_asset_download.zip"
Expand-Archive -Path "muse2_asset_download.zip" -DestinationPath muse2_bin -Force
$exePath = "${{ github.workspace }}\muse2_bin\${{ inputs.muse2_exe }}"
echo "MUSE2_PATH=$exePath" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
68 changes: 68 additions & 0 deletions .github/actions/verify-muse2/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Verify MUSE2 installation
description: Validate that muse2 is available via PATH or MUSE2_PATH for the current OS
runs:
using: composite
steps:
- name: Verify muse2 location (Linux / macOS)
if: runner.os == 'Linux' || runner.os == 'macOS'
shell: bash
run: |
set -euo pipefail

path_from_path="$(command -v muse2 || true)"
resolved_muse2="${path_from_path:-${MUSE2_PATH:-}}"

if [[ -z "$resolved_muse2" ]]; then
echo "::error::Could not find muse2 on PATH and MUSE2_PATH is not set"
exit 1
fi

if [[ ! -f "$resolved_muse2" ]]; then
echo "::error::MUSE2 executable not found at: $resolved_muse2"
exit 1
fi

if [[ ! -x "$resolved_muse2" ]]; then
echo "::error::MUSE2 file at $resolved_muse2 is not executable"
exit 1
fi

if [[ -n "$path_from_path" ]]; then
echo "Found muse2 on PATH: $resolved_muse2"
else
echo "Using MUSE2_PATH: $resolved_muse2"
fi

echo "MUSE2_PATH=$resolved_muse2" >> "$GITHUB_ENV"

- name: Verify muse2 location (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
$pathCommand = Get-Command muse2 -ErrorAction SilentlyContinue
$resolvedMuse2 = if ($pathCommand) { $pathCommand.Source } else { $env:MUSE2_PATH }

if ([string]::IsNullOrWhiteSpace($resolvedMuse2)) {
Write-Error "Could not find muse2 on PATH and MUSE2_PATH is not set"
exit 1
}

if (-not (Test-Path -Path $resolvedMuse2 -PathType Leaf)) {
Write-Error "MUSE2 executable not found at: $resolvedMuse2"
exit 1
}

$extension = [System.IO.Path]::GetExtension($resolvedMuse2).ToLowerInvariant()
if ($extension -ne '.exe') {
Write-Error "MUSE2 file at $resolvedMuse2 does not have an executable extension"
exit 1
}

if ($pathCommand) {
Write-Host "Found muse2 on PATH: $resolvedMuse2"
}
else {
Write-Host "Using MUSE2_PATH: $resolvedMuse2"
}

"MUSE2_PATH=$resolvedMuse2" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append
128 changes: 16 additions & 112 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,120 +1,24 @@
name: Test and build
name: Test against MUSE2
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR/issue description mentions running the MUSE2-main build periodically, but ci.yml doesn't add a schedule trigger (it only runs on push/PR/manual). Either add a cron schedule for the main-branch job or adjust the description/issue expectations.

Copilot uses AI. Check for mistakes.
on:
push:
branches: [main]
pull_request:
schedule:
- cron: 0 5 * * 1
workflow_dispatch:
workflow_call:

jobs:
test:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
python-version: '3.14'
muse2_asset: muse2_linux.tar.gz
muse2_exe: muse2
- os: windows-latest
python-version: '3.14'
muse2_asset: muse2_windows.zip
muse2_exe: muse2.exe
- os: macos-latest
python-version: '3.14'
muse2_asset: muse2_macos_arm.tar.gz
muse2_exe: muse2

steps:
- uses: actions/checkout@v6

- uses: astral-sh/setup-uv@v7
with:
enable-cache: true
prune-cache: false
activate-environment: true
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: uv sync

# Query the GitHub API for the latest MUSE2 release tag (e.g. "v2.0.0").
# The tag is written to GITHUB_OUTPUT so subsequent steps can reference it.
- name: Get latest MUSE2 release tag
id: muse2_release
shell: bash
run: |
TAG=$(curl -sSf \
-H "Accept: application/vnd.github+json" \
-H "Authorization: Bearer ${{ github.token }}" \
https://api.github.com/repos/EnergySystemsModellingLab/MUSE2/releases/latest \
| jq -r '.tag_name')

if [[ -z "$TAG" || "$TAG" == "null" ]]; then
echo "::error::Failed to retrieve latest MUSE2 release tag."
exit 1
fi

echo "Resolved latest MUSE2 release: $TAG"
echo "tag=$TAG" >> "$GITHUB_OUTPUT"

# Download the platform-appropriate MUSE2 release asset.
# 'shell: bash' is used on all platforms
- name: Download MUSE2 release asset
shell: bash
run: |
curl -sSfL \
"https://github.com/EnergySystemsModellingLab/MUSE2/releases/download/${{ steps.muse2_release.outputs.tag }}/${{ matrix.muse2_asset }}" \
--output "muse2_asset_download"

- name: Extract MUSE2 and set environment variable (Linux / macOS)
if: runner.os == 'Linux' || runner.os == 'macOS'
shell: bash
run: |
mkdir -p muse2_bin
tar -xf muse2_asset_download -C muse2_bin
chmod +x "muse2_bin/${{ matrix.muse2_exe }}"
echo "MUSE2_PATH=${{ github.workspace }}/muse2_bin/${{ matrix.muse2_exe }}" >> "$GITHUB_ENV"

- name: Extract MUSE2 and set environment variable (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
New-Item -ItemType Directory -Force -Path muse2_bin | Out-Null
Rename-Item -Path "muse2_asset_download" -NewName "muse2_asset_download.zip"
Expand-Archive -Path "muse2_asset_download.zip" -DestinationPath muse2_bin -Force
$exePath = "${{ github.workspace }}\muse2_bin\${{ matrix.muse2_exe }}"
echo "MUSE2_PATH=$exePath" | Out-File -FilePath $env:GITHUB_ENV -Encoding utf8 -Append

# Confirm the muse2 executable is present and the environment variable is set correctly.
- name: Verify MUSE2 installation (Linux / macOS)
if: runner.os == 'Linux' || runner.os == 'macOS'
shell: bash
run: |
if [[ ! -f "$MUSE2_PATH" ]]; then
echo "::error::MUSE2 executable not found at: $MUSE2_PATH"
exit 1
fi
echo "MUSE2_PATH is set to: $MUSE2_PATH"

- name: Verify MUSE2 installation (Windows)
if: runner.os == 'Windows'
shell: pwsh
run: |
if (-not (Test-Path -Path $env:MUSE2_PATH -PathType Leaf)) {
Write-Error "MUSE2 executable not found at: $env:MUSE2_PATH"
exit 1
}
Write-Host "MUSE2_PATH is set to: $env:MUSE2_PATH"

- name: Run tests
run: pytest

- name: Upload coverage to Codecov
if: runner.os == 'Linux'
uses: codecov/codecov-action@v6
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.xml
fail_ci_if_error: true
test_release:
name: Latest MUSE2 release
uses: ./.github/workflows/test-with-muse2.yml
with:
muse2_source: release
secrets: inherit

test_main:
name: Current MUSE2 main branch
uses: ./.github/workflows/test-with-muse2.yml
with:
muse2_source: main
secrets: inherit
Comment on lines +19 to +24
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a full “build MUSE2 from main” test job to the standard CI on every push/PR, but the PR description/issue calls out running the main-branch build periodically. If the intent is periodic coverage, consider gating test_main behind a schedule/manual trigger (or making it conditional) to avoid significantly increasing CI duration and external network/build load on every PR.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

@Aurashk Aurashk Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you probably need this in PR's too though?

Copy link
Copy Markdown
Member

@alexdewar alexdewar Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we def want it for PRs, but it would be good to also run this every week or so. We have an issue for this (#43) and I've taken the liberty of assigning you to it just now 😉.

You just need to add a cron trigger for the workflow.

Copy link
Copy Markdown
Member

@alexdewar alexdewar Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you do this in this PR while you're at it? It's just a small change to this file.

69 changes: 69 additions & 0 deletions .github/workflows/test-with-muse2.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
name: Test with MUSE2

on:
workflow_call:
inputs:
muse2_source:
description: Which MUSE2 source to test against (release or main)
required: true
type: string
secrets:
CODECOV_TOKEN:
required: true

Comment on lines +10 to +13
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CODECOV_TOKEN is marked as a required secret for the reusable workflow. Because required workflow_call secrets are enforced at invocation time (even when the Codecov step is skipped), this makes all callers (including the muse2_source: main run) require the token and will cause CI to fail for PRs from forks where secrets aren’t available. Consider making CODECOV_TOKEN optional and additionally gating the Codecov upload step on the token being present (or moving coverage upload to a separate workflow that’s only invoked for release).

Copilot uses AI. Check for mistakes.
jobs:
test:
name: Run Pytest
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
python-version: '3.14'
muse2_exe: muse2
- os: windows-latest
python-version: '3.14'
muse2_exe: muse2.exe
- os: macos-latest
python-version: '3.14'
muse2_exe: muse2

steps:
- uses: actions/checkout@v6

- uses: astral-sh/setup-uv@v7
with:
enable-cache: true
prune-cache: false
activate-environment: true
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: uv sync

- name: Setup MUSE2 from main
if: inputs.muse2_source == 'main'
uses: ./.github/actions/setup-muse2-main

- name: Setup MUSE2 from latest release
if: inputs.muse2_source == 'release'
uses: ./.github/actions/setup-muse2-release
Comment on lines +45 to +51
Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

muse2_source controls which setup step runs, but the workflow never validates the input. Any unexpected value will skip both setup steps and then fail in verify-muse2 with a misleading “executable not found” error. Add an explicit validation step early in the job to enforce allowed values (e.g. main/release) and emit a clear error message before proceeding.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor Author

@Aurashk Aurashk Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there doesn't seem to be a nice way to do this within actions scripts, and I don't think it's super likely to cause an issue

Copy link
Copy Markdown
Member

@alexdewar alexdewar Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is fine.

with:
github_token: ${{ github.token }}
muse2_exe: ${{ matrix.muse2_exe }}

- name: Verify MUSE2 installation
uses: ./.github/actions/verify-muse2

- name: Run tests
run: pytest

- name: Upload coverage to Codecov
# Latest linux release only - they should all be the same
if: runner.os == 'Linux' && inputs.muse2_source == 'release'
Comment thread
Aurashk marked this conversation as resolved.
uses: codecov/codecov-action@v6
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: ./coverage.xml
fail_ci_if_error: true
Comment thread
Aurashk marked this conversation as resolved.
Loading