feat: Actions Fleet Monitor — pure telemetry, shim interface, docs, lint

[don-petry]

Summary

• Strips Claude / Node from the daily PR review health check; replaces with pure gh/jq telemetry (failure rate, duration percentiles, per-run table written to Step Summary + Issue)
• Renames workflow file and name: from daily-pr-review-health → actions-fleet-monitor
• Adds on: workflow_call so any repo can invoke this as a shared reusable workflow
• Adds example caller stub for org-wide rollout
• Adds shellcheck lint workflow
• Adds docs/actions-fleet-monitor.md

Commits

SHA	Message
fcedcc7	refactor: replace Claude analysis with pure gh/jq telemetry
041f318	refactor: drop Node/Claude steps from workflow
0a8c732	rename: daily-pr-review-health → actions-fleet-monitor (create)
3b50d87	rename: daily-pr-review-health → actions-fleet-monitor (delete old)

Remaining work items completed in follow-up commits on this branch.

Test plan

• shellcheck passes on scripts/pr_review_health.sh
• workflow_dispatch smoke run on actions-fleet-monitor.yml completes and produces a Step Summary
• Step Summary contains the expected tables (Summary + Runs)
• Issue is created when HAS_FAILURES=true; no issue on clean run
• Example caller stub YAML is valid

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features

  • Added shell script linting checks that run on pull requests affecting scripts/ and workflows.
  • Added documentation for the Actions Fleet Monitor, detailing GitHub Actions workflow telemetry reporting and health metrics.

• Chores

  • Removed the Daily PR Review Health Check workflow and associated scripts.

[coderabbitai]

Warning

Rate limit exceeded

@don-petry has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 41 minutes and 57 seconds before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1f9c6c20-8411-4c2f-af74-14c79fc20934

📥 Commits

Reviewing files that changed from the base of the PR and between fa29235 and e953810.

⛔ Files ignored due to path filters (1)

• tests/fixtures/metrics_sample.tsv is excluded by !**/*.tsv

📒 Files selected for processing (6)

• .github/workflows/actions-fleet-monitor.yml
• .github/workflows/lint.yml
• docs/actions-fleet-monitor.md
• scripts/fleet_monitor.sh
• scripts/fleet_report.sh
• tests/fleet_report.bats

📝 Walkthrough

Walkthrough

This PR removes the legacy daily PR review health check (workflow and script) and replaces it with a new fleet monitoring system. The fleet monitor discovers org repositories and workflows, computes failure rates and percentile durations from recent runs, and generates a markdown report. A new shellcheck linting workflow validates shell scripts on pull requests.

Changes

Fleet Monitoring System

Layer / File(s)	Summary
Fleet Monitor Documentation docs/actions-fleet-monitor.md	Comprehensive documentation describing the fleet monitor's purpose, measured metrics (success/failure/cancelled counts, failure rates, p50/p95 durations), output delivery via Step Summary and GitHub Issues, configuration inputs, environment variables, and failure-rate health thresholds.
Fleet Monitor Initialization & Data Discovery scripts/fleet_monitor.sh	Script initializes with env vars (ORG, LOOKBACK_DAYS), validates gh CLI access to the org, computes the lookback cutoff timestamp, discovers all non-archived repositories via paginated API calls, and defines helper functions for severity labeling and duration formatting.
Fleet Monitor Metrics & Failed Runs Collection scripts/fleet_monitor.sh	Core loop iterates over repositories and active workflows, queries recent runs within the lookback window, calculates per-workflow success/failure/cancelled counts and failure-rate percentages, derives severity labels, computes p50/p95 durations, and collects details of failed runs with GitHub links and durations.
Fleet Monitor Report Generation & GitHub Integration scripts/fleet_monitor.sh	Generates markdown report file with a header, sorted fleet summary table, and conditionally appends "Failed Runs" section; appends report to GITHUB_STEP_SUMMARY, exports HAS_FAILURES=true to GITHUB_ENV when failures are detected, and outputs completion status.

Shell Script Linting

Layer / File(s)	Summary
Shellcheck Linting Workflow .github/workflows/lint.yml	New pull request workflow triggered by changes to scripts/** or .github/workflows/**; runs shellcheck on all shell scripts with a 5-minute timeout on Ubuntu.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main changes: replacing the PR review health check with an Actions Fleet Monitor using pure telemetry, adding a shim interface, documentation, and linting.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/actions-fleet-monitor

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[Copilot]

Pull request overview

Refactors the existing daily PR-review health check into a pure gh/jq telemetry script (removing the Claude/Node analysis path) and renames the workflow from daily-pr-review-health to actions-fleet-monitor. The new script computes failure rate and duration percentiles and emits a structured markdown report to GITHUB_STEP_SUMMARY and a file consumed by the existing issue-creation step.

Changes:

• Replaces Claude-based log analysis with a deterministic gh/jq aggregation (totals, failure rate, duration percentiles, per-run table).
• Drops Node setup, Claude CLI install, and the 60 KB report-truncation step from the workflow.
• Renames workflow file/name:/concurrency group to actions-fleet-monitor and tightens timeout-minutes from 20 → 5.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

File	Description
scripts/pr_review_health.sh	Rewrites the script as pure telemetry: computes counts, failure rate, min/p50/p95/max durations, and renders a markdown report.
.github/workflows/actions-fleet-monitor.yml	Renames workflow, removes Node/Claude install and post-truncation steps, lowers timeout to 5 minutes.

Comments suppressed due to low confidence (9)

.github/workflows/actions-fleet-monitor.yml:13

• The PR description claims this change "Adds on: workflow_call so any repo can invoke this as a shared reusable workflow", but the workflow only defines schedule and workflow_dispatch triggers — there is no workflow_call: section. As written, this workflow cannot be invoked as a reusable workflow from other repos. Either add the workflow_call: trigger (with appropriate inputs/secrets) or update the PR description.
  .github/workflows/actions-fleet-monitor.yml:1
• The PR description also lists "Adds example caller stub for org-wide rollout", "Adds shellcheck lint workflow", and "Adds docs/actions-fleet-monitor.md", but none of those files appear in this diff (verified: no docs/actions-fleet-monitor.md, no shellcheck workflow under .github/workflows/, no caller stub). Either include those files or trim the PR description so it accurately reflects what is being merged.

This issue also appears on line 3 of the same file.
scripts/pr_review_health.sh:157

• duration_s is computed as updated_at - created_at for every run, including runs whose conclusion is still null (queued/in_progress). Those rows are then included in the "Runs" table (line 155–157 does not filter by conclusion) with a duration that is really "time since the run started", not actual run time. The aggregate percentiles correctly filter conclusion != null, but the per-run table will surface misleading numbers (e.g. an in-progress run that started 23h ago will show as a ~23h run). Consider filtering in-progress runs out of the table or rendering their duration as —/in progress.

  fi

  printf '\n## Runs\n\n'
  printf '| Run | Status | Date | Duration | Link |\n|---|---|---|---|---|\n'
  while IFS=$'\t' read -r run_num conclusion created_at dur_s url; do
    icon=$(conclusion_icon "$conclusion")
    date_short="${created_at%%T*}"
    printf '| #%s | %s %s | %s | %s | [view](%s) |\n' \
      "$run_num" "$icon" "$conclusion" "$date_short" "$(fmt_dur "$dur_s")" "$url"
  done < <(echo "$runs_json" | jq -r '

scripts/pr_review_health.sh:124

• The "WARNING" bucket is only reachable when failed_runs > 0 and failure_rate ≤ 20%. As soon as there is at least one failure, the overall status jumps from HEALTHY to WARNING even if failure_rate rounds to 0.0% (e.g. 1 failure in 1000 runs). The naming/severity threshold is a judgment call, but the gap between HEALTHY (0 failures) and WARNING (any single failure) is worth documenting explicitly in the report or near these thresholds — otherwise readers can't tell what WARNING means.

}

if [ "$failed_runs" -eq 0 ]; then
  overall="HEALTHY"
elif [ "$(echo "$failure_rate > 50" | bc)" -eq 1 ]; then
  overall="CRITICAL"
elif [ "$(echo "$failure_rate > 20" | bc)" -eq 1 ]; then
  overall="DEGRADED"
else

scripts/pr_review_health.sh:154

• fmt_dur is called with unquoted argument (e.g. $(fmt_dur $dur_min)). Although dur_min/dur_p50/dur_p95/dur_max/dur_s are produced by jq and should be plain integers, if jq ever emits an unexpected value (empty string, float, decimal) the [ "$s" -ge 60 ] test or $((s / 60)) arithmetic will exit the script under set -euo pipefail. Quoting the argument ("$dur_min") and/or adding a numeric-guard fallback would make the report resilient to upstream API quirks. Same concern for the value passed in the while read loop at line 154.

  printf '| Failure rate | %s%% |\n' "$failure_rate"
  if [ "$total_runs" -gt 0 ]; then
    printf '| Duration min | %s |\n' "$(fmt_dur "$dur_min")"
    printf '| Duration p50 | %s |\n' "$(fmt_dur "$dur_p50")"
    printf '| Duration p95 | %s |\n' "$(fmt_dur "$dur_p95")"
    printf '| Duration max | %s |\n' "$(fmt_dur "$dur_max")"
  fi

  printf '\n## Runs\n\n'
  printf '| Run | Status | Date | Duration | Link |\n|---|---|---|---|---|\n'
  while IFS=$'\t' read -r run_num conclusion created_at dur_s url; do
    icon=$(conclusion_icon "$conclusion")
    date_short="${created_at%%T*}"

scripts/pr_review_health.sh:92

• The percentile expression uses integer-index $d[$n * 50 / 100 | floor]. For $n = 1 both p50 and p95 resolve to index 0, which is also the min/max — so all four columns show the same number. That's mathematically defensible but visually confusing in the Step Summary. Consider hiding the percentile rows (or just min/max) when length < 2 (or some other small-N threshold) to avoid implying meaningful percentile data when there isn't any.

# Duration percentiles across all completed runs
read -r dur_min dur_p50 dur_p95 dur_max < <(echo "$runs_json" | jq -r '
  [.[] | select(.conclusion != null and .duration_s > 0) | .duration_s] | sort |
  if length == 0 then "0 0 0 0"
  else . as $d | ($d | length) as $n |

scripts/pr_review_health.sh:158

• When total_runs == 0 the report still emits the Summary table without duration rows but with Failure rate | 0.0%. The "Runs" section header is also printed with an empty table body, which renders as an invalid/empty markdown table on GitHub. Consider rendering an explicit "No runs in lookback window" message in both Summary and Runs sections when total_runs == 0, and skipping the issue-creation path entirely.

# 4. Build report
# ---------------------------------------------------------------------------
{
  printf '# Actions Fleet Monitor — %s\n\n' "$TODAY"
  printf '**Workflow:** `%s` in `%s` | **Status:** `%s` | **Lookback:** %s day(s)\n\n' \
    "$WORKFLOW_FILE" "$WORKFLOW_REPO" "$overall" "$LOOKBACK_DAYS"

  printf '## Summary\n\n'
  printf '| Metric | Value |\n|---|---|\n'
  printf '| Total runs | %s |\n' "$total_runs"
  printf '| Successful | %s |\n' "$success_runs"
  printf '| Failed | %s |\n' "$failed_runs"
  printf '| Cancelled | %s |\n' "$cancelled_runs"
  printf '| Failure rate | %s%% |\n' "$failure_rate"
  if [ "$total_runs" -gt 0 ]; then
    printf '| Duration min | %s |\n' "$(fmt_dur "$dur_min")"
    printf '| Duration p50 | %s |\n' "$(fmt_dur "$dur_p50")"
    printf '| Duration p95 | %s |\n' "$(fmt_dur "$dur_p95")"
    printf '| Duration max | %s |\n' "$(fmt_dur "$dur_max")"
  fi

  printf '\n## Runs\n\n'
  printf '| Run | Status | Date | Duration | Link |\n|---|---|---|---|---|\n'
  while IFS=$'\t' read -r run_num conclusion created_at dur_s url; do
    icon=$(conclusion_icon "$conclusion")
    date_short="${created_at%%T*}"
    printf '| #%s | %s %s | %s | %s | [view](%s) |\n' \
      "$run_num" "$icon" "$conclusion" "$date_short" "$(fmt_dur "$dur_s")" "$url"
  done < <(echo "$runs_json" | jq -r '
    sort_by(.run_number) | reverse[] |

.github/workflows/actions-fleet-monitor.yml:26

• timeout-minutes was tightened from 20 to 5. The remaining work is a single gh api call plus jq processing, which is normally well under a minute — but gh api can occasionally retry on transient 5xx/abuse-rate-limit responses, and the previous bash step also occasionally pulled per-run data. 5 minutes leaves very little headroom for tail latency or for someone bumping LOOKBACK_DAYS to a large value (which can produce a larger paginated response in the future). 10 minutes would be a safer floor.
  scripts/pr_review_health.sh:53
• The gh api URL embeds the unsanitised ${CUTOFF} via created=>=${CUTOFF}. CUTOFF is produced by date -u so it should always be a valid ISO-8601 string, but the >= characters are not URL-encoded. gh api typically forwards the path as-is; the GitHub Actions Search API accepts the literal >=, so this works today. Worth a brief inline comment so a future maintainer doesn't try to "fix" it by URL-encoding (which would break the operator).

echo "Fetching runs since: $CUTOFF"

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[gemini-code-assist]

Code Review

This pull request refactors the pr_review_health.sh script, shifting its focus from log analysis via Claude to telemetry-based health monitoring. Key changes include the removal of log-downloading logic, the addition of aggregate statistics calculation (such as failure rates and duration percentiles), and the generation of a structured markdown report for the GitHub Actions step summary. Feedback suggests improving the script's reusability by defaulting to the current repository environment variable and recommends using API pagination to ensure all workflow runs are captured for accurate metrics.

[sonarqubecloud]

❌ The last analysis has failed.

See analysis details on SonarQube Cloud

[don-petry]

No description provided.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[don-petry]

@donpetry-bot - please review

[donpetry-bot]

Automated review — APPROVED ✓

Risk: MEDIUM
Reviewed commit: e9538100cbaf1c809407bd25f239c2a2a9d18b79
Review mode: triage-approved (single reviewer)

Summary

This PR refactors the daily PR-review health check into a generalized, org-wide Actions Fleet Monitor delivered as a reusable workflow. The Claude/Node analysis path is replaced with pure gh/jq telemetry — discovering all non-archived repos in the org, collecting per-workflow run metrics, and producing a tiered scorecard (HEALTHY / WARNING / DEGRADED / CRITICAL), Mermaid pie + bar charts, per-repo rollup, systemic-failure detection, and an ASCII bar visualization. New scripts/fleet_report.sh is unit-tested with 40 bats cases; shellcheck runs in CI on the new scripts.

Triage flagged this as low-risk auto-approve. Confirmation review agrees: the change is additive (new scripts replacing an obsolete one), covered by tests and a lint job, and properly reviewed by automated reviewers.

Linked issue analysis

No closingIssuesReferences are set, but the PR body cites Discussion #193 as the design RFC and the commit history reflects an iterative implementation against that discussion. No action item appears unaddressed.

Findings

No blocking issues.

Notes (non-blocking):

• The reusable-workflow checkout step in .github/workflows/actions-fleet-monitor.yml pins ref: main for petry-projects/.github-private (intentional per inline comment, so workflow_call callers always run the agent's main-branch code). An earlier commit (9ed164d) attempted to pin to github.workflow_sha; the final form uses main, which is consistent with the documented usage pattern (@main) in docs/actions-fleet-monitor.md. This is a reasonable trade-off — worth being aware of if you later want immutable-per-call versioning.
• The script uses DON_PETRY_BOT_GH_PAT for cross-org actions:read. This is documented in the script header and in docs/actions-fleet-monitor.md, with rationale (default GITHUB_TOKEN lacks org-wide actions:read). A GH_PAT_FALLBACK is also wired up for graceful degradation.
• Shellcheck is scoped to fleet_monitor.sh and fleet_report.sh and runs with --severity=warning (info-level SC2016 from literal markdown backticks suppressed); justification is in the workflow comment.
• Issue creation on failure uses github.token + context.repo, which means caller-invoked runs create issues in the caller's repo — explicitly documented in the "Issue destination" section of the docs.

CI status

All required checks green:

• agent-shield / AgentShield ✅
• Analyze (actions) (CodeQL) ✅
• dependency-audit / Detect ecosystems ✅ (per-ecosystem children skipped — none present)
• dispatch (Dev-Lead Agent) ✅
• shellcheck (Lint) ✅
• bats (Lint) ✅
• unit-tests ✅
• SonarCloud ✅, SonarCloud Code Analysis ✅
• CodeRabbit ✅

Mergeable: MERGEABLE / CLEAN. Review decision: APPROVED.

---

Reviewed automatically by the PR-review agent (single-reviewer mode: opus 4.7). Reply if you need a human review.