-
Notifications
You must be signed in to change notification settings - Fork 424
Compile: move checkout-manifest generation to github-script to unblock dynamic checkout.repository expressions #38154
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
a9de0a0
Initial plan
Copilot 844878b
Plan: fix checkout manifest template injection handling
Copilot a5e69ca
Fix checkout manifest step template-injection regression
Copilot 45bcda4
chore: plan follow-up for review feedback
Copilot d3d22f4
Merge branch 'main' into copilot/fix-compile-failure-v0783
github-actions[bot] 71f094a
Add checkout manifest logging and shim core usage
Copilot 627d66b
Add checkout-credential-review skill and wire from AGENTS.md
dsyme 5a208c9
Use per-checkout tokens for checkout-manifest gh fallback
Copilot c195af0
Merge branch 'main' into copilot/fix-compile-failure-v0783
pelikhan ba29dbf
Use per-checkout tokens in checkout manifest fallback and recompile
Copilot 1da46ef
Merge branch 'main' into copilot/fix-compile-failure-v0783
pelikhan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| --- | ||
| name: checkout-credential-review | ||
| description: Review code that performs git or gh operations against repository checkouts in gh-aw, checking that the right credentials are available at the right time and that sparseness, shallowness and credential-free factors are properly considered. | ||
| --- | ||
|
|
||
| # Checkout Credential Review | ||
|
|
||
| Use this skill when reviewing or writing code in `pkg/workflow/`, `actions/setup/js/`, or compiled `.lock.yml` workflows that runs `git`, `gh`, or any other remote-touching operation against a repository checkout. | ||
|
|
||
| ## Background | ||
|
|
||
| Each entry in a workflow's `checkout:` block may declare its own credentials (`github-token:`, `github-app:`), and the compiler wires those into the corresponding `actions/checkout` step ([pkg/workflow/checkout_step_generator.go](pkg/workflow/checkout_step_generator.go)). Generated checkouts always set `persist-credentials: false`, so the on-disk repo retains **no** credentials after the step finishes — only `actions/checkout`'s own internal token is used during the clone, and it is scrubbed in its post-step. | ||
|
|
||
| A separate step that wants to authenticate later must either (a) re-inject a token at command level (e.g. `git -c http.extraheader=...`) or (b) be passed the per-checkout token via env. The compiler does *not* automatically thread per-checkout `github-token`s into downstream steps. | ||
|
|
||
| Two important contexts deliberately run with **no git credentials**: | ||
|
|
||
| - The **safe-outputs MCP server** and its handlers (`generate_git_bundle.cjs`, `generate_git_patch.cjs`, `create_pull_request.cjs`). Errors in these paths explicitly say "the safe-outputs MCP server has no credentials for private repositories" — fetch/push will fail for private repos. | ||
| - The **agent runtime** after `actions/checkout`. The agent prompt in [actions/setup/md/safe_outputs_push_to_pr_branch.md](actions/setup/md/safe_outputs_push_to_pr_branch.md) explicitly tells the model not to attempt `git fetch`, `git pull`, `git push`, or any other authenticated git operation, and to report unavailable branches rather than try to fetch them. | ||
|
|
||
| ## Review checklist | ||
|
|
||
| When you see a new `git`, `gh`, `execFileSync('git'…)`, or compiled `run:` block: | ||
|
|
||
| 1. **Does it touch a remote?** Local-only commands (`symbolic-ref`, `rev-parse`, `log`, `show`, `merge-base`, `diff`, `status`) need no credentials. Anything in `fetch | pull | push | clone | ls-remote | remote (set-url|add|update)` does, plus on-demand blob fetches in partial clones. | ||
| 2. **Which checkout is it operating on?** If it's a cross-repo entry from `checkout:`, the relevant credential is *that entry's* `github-token`, not the workflow's default `GITHUB_TOKEN`. Confirm the per-entry token is actually threaded into the step's env (or refuse to do remote operations and degrade gracefully). | ||
| 3. **Which job/context emits it?** Agent job and safe-outputs MCP server both run without git credentials by design. Any remote git operation there must be wrapped in `try/catch`, fail soft, and surface a clear "no credentials" error rather than a raw git stderr. | ||
| 4. **Sparse / shallow / monorepo concerns.** Avoid emitting steps that deepen (`git fetch --unshallow`, `--deepen=N`) or widen (`git fetch origin '+refs/heads/*'`) a sparse or shallow checkout of a large monorepo — these need credentials *and* can pull hundreds of MB. Prefer expanding `fetch:` / `fetch-depth:` / `sparse-checkout:` at compile time so it happens during `actions/checkout` with its internal token, never later. | ||
| 5. **`gh` is REST, not git.** `gh api …` uses whatever `GH_TOKEN` is in the step's env — it does **not** automatically inherit per-checkout PATs. For cross-org private repos, either thread the right token in or accept the call will 404 and handle it. | ||
|
|
||
| ## Related | ||
|
|
||
| - [docs/src/content/docs/reference/checkout.md](docs/src/content/docs/reference/checkout.md) — "Git Credentials After Checkout" | ||
| - [docs/sparseness.md](docs/sparseness.md) — sparse/blobless credential lifecycle | ||
| - [pkg/workflow/checkout_step_generator.go](pkg/workflow/checkout_step_generator.go) — token wiring per checkout | ||
| - [actions/setup/md/safe_outputs_push_to_pr_branch.md](actions/setup/md/safe_outputs_push_to_pr_branch.md) — agent-facing guidance |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,143 @@ | ||
| // @ts-check | ||
| /// <reference types="@actions/github-script" /> | ||
|
|
||
| require("./shim.cjs"); | ||
|
|
||
| const fs = require("fs"); | ||
| const path = require("path"); | ||
| const { execFileSync } = require("child_process"); | ||
|
|
||
| const { getErrorMessage } = require("./error_helpers.cjs"); | ||
|
|
||
| function parseManifestEntries(entriesJSON = process.env.GH_AW_CHECKOUT_MANIFEST_ENTRIES || "[]") { | ||
| const parsed = JSON.parse(entriesJSON); | ||
| if (!Array.isArray(parsed)) { | ||
| throw new Error("GH_AW_CHECKOUT_MANIFEST_ENTRIES must be a JSON array"); | ||
| } | ||
| return parsed; | ||
| } | ||
|
|
||
| function readManifestEntriesFromEnv() { | ||
| const count = Number.parseInt(process.env.GH_AW_CHECKOUT_MANIFEST_COUNT || "0", 10); | ||
| if (!Number.isFinite(count) || count < 0) { | ||
| throw new Error("GH_AW_CHECKOUT_MANIFEST_COUNT must be a non-negative integer"); | ||
| } | ||
|
|
||
| const entries = []; | ||
| for (let i = 0; i < count; i += 1) { | ||
| entries.push({ | ||
| repository: process.env[`GH_AW_CHECKOUT_REPO_${i}`] || "", | ||
| path: process.env[`GH_AW_CHECKOUT_PATH_${i}`] || "", | ||
| token: process.env[`GH_AW_CHECKOUT_TOKEN_${i}`] || "", | ||
| }); | ||
| } | ||
| return entries; | ||
| } | ||
|
|
||
| function resolveDefaultBranch(repository, checkoutPath, options = {}) { | ||
| const workspace = options.workspace || process.env.GITHUB_WORKSPACE || ""; | ||
| const runGit = options.runGit || ((args, execOptions = {}) => execFileSync("git", args, { encoding: "utf8", ...execOptions })); | ||
| const runGH = | ||
| options.runGH || | ||
| ((args, execOptions = {}) => | ||
| execFileSync("gh", args, { | ||
| encoding: "utf8", | ||
| env: { ...process.env, ...(execOptions.env || {}) }, | ||
| ...execOptions, | ||
| })); | ||
| let defaultBranch = ""; | ||
|
|
||
| const repoPath = checkoutPath ? path.join(workspace, checkoutPath) : workspace; | ||
| if (repoPath && fs.existsSync(path.join(repoPath, ".git"))) { | ||
| try { | ||
| const output = runGit(["-C", repoPath, "symbolic-ref", "--short", "refs/remotes/origin/HEAD"], { | ||
| stdio: ["ignore", "pipe", "pipe"], | ||
| }); | ||
| defaultBranch = output.trim().replace(/^origin\//, ""); | ||
| core.debug(`build_checkout_manifest: git resolved default branch for ${repository}: ${defaultBranch}`); | ||
| } catch (error) { | ||
| core.debug(`build_checkout_manifest: git default branch lookup failed for ${repository}: ${getErrorMessage(error)}`); | ||
| } | ||
| } | ||
|
|
||
| if (defaultBranch === "") { | ||
| try { | ||
| const checkoutToken = options.checkoutToken || ""; | ||
| const ghExecOptions = { | ||
| stdio: ["ignore", "pipe", "pipe"], | ||
| }; | ||
| if (checkoutToken !== "") { | ||
| ghExecOptions.env = { GH_TOKEN: checkoutToken }; | ||
| } | ||
| defaultBranch = runGH(["api", `repos/${repository}`, "--jq", ".default_branch"], ghExecOptions).trim(); | ||
| core.debug(`build_checkout_manifest: gh api resolved default branch for ${repository}: ${defaultBranch}`); | ||
| } catch (error) { | ||
| core.debug(`build_checkout_manifest: gh api default branch lookup failed for ${repository}: ${getErrorMessage(error)}`); | ||
| } | ||
| } | ||
|
|
||
| return defaultBranch; | ||
| } | ||
|
|
||
| function buildCheckoutManifest(entries, options = {}) { | ||
| const runnerTemp = options.runnerTemp || process.env.RUNNER_TEMP; | ||
| if (!runnerTemp) { | ||
| throw new Error("RUNNER_TEMP is required to build checkout manifest"); | ||
| } | ||
|
|
||
| const runGit = options.runGit; | ||
| const runGH = options.runGH; | ||
|
|
||
| const manifestDir = path.join(runnerTemp, "gh-aw"); | ||
| fs.mkdirSync(manifestDir, { recursive: true }); | ||
| const manifestPath = path.join(manifestDir, "checkout-manifest.json"); | ||
| const manifest = {}; | ||
| core.info(`checkout-manifest: building manifest for ${entries.length} checkout entries`); | ||
|
|
||
| for (const entry of entries) { | ||
| if (!entry || typeof entry !== "object") { | ||
| core.debug("checkout-manifest: skipping non-object entry"); | ||
| continue; | ||
| } | ||
| const repository = String(entry.repository || "").trim(); | ||
| if (repository === "") { | ||
| core.debug("checkout-manifest: skipping entry with empty repository"); | ||
| continue; | ||
| } | ||
| const checkoutPath = String(entry.path || ""); | ||
| const defaultBranch = resolveDefaultBranch(repository, checkoutPath, { | ||
| workspace: options.workspace, | ||
| runGit, | ||
| runGH, | ||
| checkoutToken: entry.token || "", | ||
| }); | ||
| manifest[repository.toLowerCase()] = { | ||
| repository, | ||
| path: checkoutPath, | ||
| default_branch: defaultBranch, | ||
| }; | ||
| core.info(`checkout-manifest: ${repository} -> path=${checkoutPath} default_branch=${defaultBranch || "<unresolved>"}`); | ||
| } | ||
|
|
||
| fs.writeFileSync(manifestPath, JSON.stringify(manifest, null, 2) + "\n", "utf8"); | ||
| core.info(`checkout-manifest written to ${manifestPath}`); | ||
| return { manifestPath, manifest }; | ||
| } | ||
|
|
||
| async function main(options = {}) { | ||
| let entries; | ||
| if (typeof options.entriesJSON === "string" && options.entriesJSON.trim() !== "") { | ||
| entries = parseManifestEntries(options.entriesJSON); | ||
| } else { | ||
| entries = readManifestEntriesFromEnv(); | ||
| } | ||
| return buildCheckoutManifest(entries, options); | ||
| } | ||
|
|
||
| module.exports = { | ||
| buildCheckoutManifest, | ||
| main, | ||
| parseManifestEntries, | ||
| readManifestEntriesFromEnv, | ||
| resolveDefaultBranch, | ||
| }; | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.