fix(ci): run AOAI api-key smoke pre-step in activation job so prompt step outputs resolve#4540
Conversation
|
@copilot instead of this, couldn't we just move Also remove unrelated otel test fix. |
Done. Moved the One note: since the activation job is a separate runner from the agent job, the file-write test's file no longer exists on the agent side, so I changed test #3 to have the agent write+read its own scratch file in Also removed the unrelated smoke-otel-tracing change. (49bf...) |
✅ Coverage Check PassedOverall Coverage
📁 Per-file Coverage Changes (2 files)
Coverage comparison generated by |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@copilot suggest PR title |
This comment has been minimized.
This comment has been minimized.
Suggested PR title:
A few shorter alternatives:
|
Smoke Test Results
Overall Status: FAIL PR titles (partial from git log):
Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
This comment has been minimized.
This comment has been minimized.
* fix: allow BYOK provider model validation * fix: check env-file for provider base URL in model bypass --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
|
@copilot rebase |
A true linear rebase rewrites history and needs a force-push, which my tooling can't do (pushes are fast-forward-only). Instead I brought the branch current with the latest |
|
GitHub MCP Testing: ✅ Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "api.openai.com"See Network Configuration for more information.
|
|
Smoke Test: Copilot BYOK (Direct Mode) — PASS ✅ Recent PRs:
Test Results:
Mode:
|
Chroot Version Comparison Results
Overall: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot.
|
Smoke Test
|
🔍 Smoke Test: Copilot PAT Auth — PASS
PR: fix(ci): run AOAI api-key smoke pre-step in activation job so prompt step outputs resolve
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
|
GitHub API: ✅ PASS Total: PASS
|
🤖 Smoke Test Results
Overall: PASS PR by
|
|
Running in direct BYOK mode (COPILOT_PROVIDER_API_KEY + COPILOT_PROVIDER_BASE_URL) via api-proxy → Azure OpenAI (Foundry, o4-mini-aw) Overall: PASS
|
Smoke Test Results — FAIL ❌
Root cause: AWF iptables rules explicitly block ports 6379 (Redis) and 5432 (PostgreSQL) in the OUTPUT chain.
|
There was a problem hiding this comment.
Pull request overview
This PR updates the Copilot BYOK AOAI api-key smoke workflow so the activation job can precompute and expose step outputs that the prompt rendering relies on, preventing empty placeholders across the activation→agent job boundary. It also adjusts COPILOT_MODEL validation behavior to permit custom model names when a Copilot provider base URL indicates BYOK routing.
Changes:
- Moved the BYOK smoke “smoke-data” precompute step into the workflow’s activation job so
${{ steps.smoke-data.outputs.* }}resolves during prompt creation. - Updated the smoke prompt’s file I/O test to be agent-runner-local (since activation and agent jobs run on different runners).
- Relaxed COPILOT_MODEL validation when a Copilot provider base URL is configured, and added unit tests for the new behavior.
Show a summary per file
| File | Description |
|---|---|
src/commands/validators/config-assembly.ts |
Adds provider base URL detection to conditionally skip COPILOT_MODEL validation; refactors env-file parsing helper. |
src/commands/validators/config-assembly.test.ts |
Adds tests ensuring custom COPILOT_MODEL values are allowed when provider base URL is present. |
.github/workflows/smoke-copilot-byok-aoai-apikey.md |
Moves smoke-data into activation pre-steps and updates the prompt instructions for file I/O. |
.github/workflows/smoke-copilot-byok-aoai-apikey.lock.yml |
Regenerates compiled workflow to reflect activation-job relocation and prompt substitutions. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 4/4 changed files
- Comments generated: 3
| const hasCustomCopilotProviderBaseUrl = !!( | ||
| config.copilotProviderBaseUrl || | ||
| config.additionalEnv?.COPILOT_PROVIDER_BASE_URL || | ||
| readEnvVarFromEnvFiles((config as { envFile?: unknown }).envFile, 'COPILOT_PROVIDER_BASE_URL') || | ||
| (config.envAll ? process.env.COPILOT_PROVIDER_BASE_URL : undefined) | ||
| ); | ||
| if ( | ||
| copilotModel && | ||
| !hasCustomCopilotProviderBaseUrl && | ||
| (config.copilotGithubToken || config.copilotProviderApiKey) | ||
| ) { |
| const hasCustomCopilotProviderBaseUrl = !!( | ||
| config.copilotProviderBaseUrl || | ||
| config.additionalEnv?.COPILOT_PROVIDER_BASE_URL || | ||
| readEnvVarFromEnvFiles((config as { envFile?: unknown }).envFile, 'COPILOT_PROVIDER_BASE_URL') || | ||
| (config.envAll ? process.env.COPILOT_PROVIDER_BASE_URL : undefined) | ||
| ); |
| Pre-step wrote and read back: "${{ steps.smoke-data.outputs.SMOKE_FILE_CONTENT }}" | ||
| File path: ${{ steps.smoke-data.outputs.SMOKE_FILE_PATH }} | ||
| Verify by running `cat` on the file path using bash to confirm it exists. | ||
| The activation pre-step wrote and read back: "${{ steps.smoke-data.outputs.SMOKE_FILE_CONTENT }}" (that file lives on the activation runner, not here). To exercise agent-side file I/O in the sandbox, write a short string to `/tmp/gh-aw/agent/agent-write-test.txt` and `cat` it back with bash. ✅ if the read-back matches. |
The
smoke-copilot-byok-aoai-apikeyworkflow fed pre-computed test data into the agent prompt via${{ steps.smoke-data.outputs.* }}, which doesn't resolve across the activation→agent job split — leaving the prompt with empty PR data, HTTP code, and file-content placeholders. The root cause is that thesmoke-datastep compiled into the agent job, while the prompt is rendered in the activation job.Changes
smoke-datato the activation job — thePre-compute BYOK smoke test datastep is now declared underjobs.activation.pre-steps, so it runs in the same built-in job where the prompt is created. As a result${{ steps.smoke-data.outputs.* }}resolves correctly. The step keeps itsid: smoke-dataand continues to emitSMOKE_PR_DATA,SMOKE_HTTP_CODE,SMOKE_FILE_CONTENT, andSMOKE_FILE_PATHvia$GITHUB_OUTPUT.${{ steps.smoke-data.outputs.* }}expressions directly (no filecat-ing handoff)./tmp/gh-aw/agent/to exercise sandbox file I/O, while the pre-step's write/read result is still surfaced throughSMOKE_FILE_CONTENT.scripts/ci/postprocess-smoke-workflows.ts; the.lock.ymlplaces the step in theactivationjob ahead of the prompt-creation step and retains--build-local.