Fix Large Payload Tester: move secret to top level, harden agent prompt by Copilot · Pull Request #7681 · github/gh-aw-mcpg

Copilot · 2026-06-17T14:01:41Z

The Large Payload Tester workflow failed intermittently when the AI agent bypassed the MCP filesystem tools and instead shelled out directly to find the test file — completely skipping the large-payload mechanism it was supposed to exercise. This burned ~1337 AIC building custom MCP infrastructure before hitting the engine timeout.

Two root causes:

1. Secret buried in a 2000-item array

secret_reference lived at data.large_array[1000].secret_reference. A natural top-level dict probe returns None, confusing the agent about what it found.

Before:

.data.large_array = [range(2000) | . as $i | if $i == 1000 then {id: $i, ..., secret_reference: $secret, ...} else {...} end]

After: secret_reference is a top-level field, set first in the jq filter chain:

'.secret_reference = $secret |
 .test_run_id = $run_id |
 .data.large_array = [range(2000) | . as $i | {id: $i, value: ..., random_data: ..., extra_data: ...}]

2. Agent prompt didn't specify the MCP tool path

The prompt never said how to read the file, so the agent fell back to find /tmp -name "large-test-file.json" in bash, finding the raw host path and never triggering the gateway's payload offloading at all.

The rewritten prompt explicitly:

Requires filesystem-read_file MCP tool (not bash) for both test files
Specifies exact container paths (/workspace/large-test-file.json, /workspace/secret.txt)
Warns against reading directly from /tmp/mcp-test-fs/
States secret_reference is at the top level of the JSON at payloadPath

Files changed:

.github/workflows/large-payload-tester.md — JSON template + jq filter
.github/workflows/large-payload-tester.lock.yml — compiled lock file updated to match
.github/agentics/large-payload-tester.md — agent prompt rewritten for clarity

…prompt

Copilot

Pull request overview

This PR updates the “Large Payload Tester” agentic workflow to make the secret easier to extract from large-payload offloaded data and to reduce flakiness by tightening the agent instructions around how files must be read (via MCP tools rather than host filesystem discovery).

Changes:

Move secret_reference from a deep location inside data.large_array[1000] to a top-level JSON field populated early in the jq pipeline.
Simplify the generated large array items (remove the special “middle item” secret injection).
Rewrite the agent prompt to explicitly require filesystem-read_file for /workspace/* inputs and document the expected payloadPath flow.

Show a summary per file

File	Description
.github/workflows/large-payload-tester.md	Updates the source workflow template’s JSON template and jq generation to place `secret_reference` at the top level.
.github/workflows/large-payload-tester.lock.yml	Regenerates the compiled workflow to reflect the updated JSON/jq generation behavior.
.github/agentics/large-payload-tester.md	Hardens the agent instructions to use MCP filesystem tools and clarifies the large-payload `payloadPath` retrieval steps.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 3/3 changed files
Comments generated: 3

          EXPR_GITHUB_SERVER_URL: ${{ github.server_url }}
        name: Setup Test Environment
-        run: "TEST_FS=\"/tmp/mcp-test-fs\"\nSECRET_FILE=\"secret.txt\"\nLARGE_PAYLOAD_FILE=\"large-test-file.json\"\n# Create test data directory (payload directory will be created by gateway on-demand)\nmkdir -p $TEST_FS\n\n# Generate a unique secret for this test run\n# Use uuidgen if available, otherwise use timestamp with nanoseconds for better entropy\nif command -v uuidgen >/dev/null 2>&1; then\n  TEST_SECRET=\"test-secret-$(uuidgen)\"\nelse\n  TEST_SECRET=\"test-secret-$(date +%s%N)-$$\"\nfi\necho $TEST_SECRET > $TEST_FS/$SECRET_FILE\n# Create a large test file (~500KB) with the secret embedded in JSON\n# This file will be read by the filesystem MCP server, causing a large payload\ncat > $TEST_FS/$LARGE_PAYLOAD_FILE <<'EOF'\n{\n  \"test_run_id\": \"PLACEHOLDER_RUN_ID\",\n  \"test_timestamp\": \"PLACEHOLDER_TIMESTAMP\",\n  \"purpose\": \"Testing large MCP payload storage and retrieval\",\n  \"data\": {\n    \"large_array\": [],\n    \"metadata\": {\n      \"generated_by\": \"large-payload-tester workflow\",\n      \"repository\": \"PLACEHOLDER_REPO\",\n      \"workflow_run_url\": \"PLACEHOLDER_URL\"\n    }\n  },\n  \"padding\": \"\"\n}\nEOF\n\n# Use jq to properly populate the JSON with dynamic values and generate large array\n# Generating 2000 items + 400KB padding to create ~500KB file\n# Secret is only included once in the middle item (index 1000)\njq --arg secret \"$TEST_SECRET\" \\\n   --arg run_id \"$EXPR_GITHUB_RUN_ID\" \\\n   --arg timestamp \"$(date -Iseconds)\" \\\n   --arg repo \"$EXPR_GITHUB_REPOSITORY\" \\\n   --arg url \"$EXPR_GITHUB_SERVER_URL/$EXPR_GITHUB_REPOSITORY/actions/runs/$EXPR_GITHUB_RUN_ID\" \\\n   '.test_run_id = $run_id | \n    .test_timestamp = $timestamp | \n    .data.metadata.repository = $repo | \n    .data.metadata.workflow_run_url = $url | \n    .data.large_array = [range(2000) | . as $i | if $i == 1000 then {id: $i, value: (\"item-\" + tostring), secret_reference: $secret, extra_data: (\"data-\" + tostring + \"-\" * 50)} else {id: $i, value: (\"item-\" + tostring), random_data: (\"rand-\" + ($i * 17 % 9973 | tostring) + \"-\" + ($i * 31 % 8191 | tostring)), extra_data: (\"data-\" + tostring + \"-\" * 50)} end] |\n    .padding = (\"X\" * 400000)' \\\n   $TEST_FS/$LARGE_PAYLOAD_FILE > $TEST_FS/$LARGE_PAYLOAD_FILE.tmp\n\nmv $TEST_FS/$LARGE_PAYLOAD_FILE.tmp $TEST_FS/$LARGE_PAYLOAD_FILE\n\n# Verify file was created and is large enough\nFILE_SIZE=$(wc -c < $TEST_FS/$LARGE_PAYLOAD_FILE)\necho \"Created $LARGE_PAYLOAD_FILE with size: $FILE_SIZE bytes (~$(($FILE_SIZE / 1024))KB)\"\nif [ \"$FILE_SIZE\" -lt 512000 ]; then\n  echo \"WARNING: Test file is smaller than expected ($FILE_SIZE bytes < 500KB)\"\n  echo \"Continuing with test anyway...\"\nfi\n\necho \"Test environment setup complete\"\necho \"Large file stored in: $TEST_FS/$LARGE_PAYLOAD_FILE\"\ngrep -H $TEST_SECRET $TEST_FS/$LARGE_PAYLOAD_FILE\necho \"Secret stored in $TEST_FS/$SECRET_FILE\"\ngrep -H $TEST_SECRET $TEST_FS/$SECRET_FILE\n"
+        run: "TEST_FS=\"/tmp/mcp-test-fs\"\nSECRET_FILE=\"secret.txt\"\nLARGE_PAYLOAD_FILE=\"large-test-file.json\"\n# Create test data directory (payload directory will be created by gateway on-demand)\nmkdir -p $TEST_FS\n\n# Generate a unique secret for this test run\n# Use uuidgen if available, otherwise use timestamp with nanoseconds for better entropy\nif command -v uuidgen >/dev/null 2>&1; then\n  TEST_SECRET=\"test-secret-$(uuidgen)\"\nelse\n  TEST_SECRET=\"test-secret-$(date +%s%N)-$$\"\nfi\necho $TEST_SECRET > $TEST_FS/$SECRET_FILE\n# Create a large test file (~500KB) with the secret embedded in JSON\n# This file will be read by the filesystem MCP server, causing a large payload\ncat > $TEST_FS/$LARGE_PAYLOAD_FILE <<'EOF'\n{\n  \"test_run_id\": \"PLACEHOLDER_RUN_ID\",\n  \"test_timestamp\": \"PLACEHOLDER_TIMESTAMP\",\n  \"secret_reference\": \"\",\n  \"purpose\": \"Testing large MCP payload storage and retrieval\",\n  \"data\": {\n    \"large_array\": [],\n    \"metadata\": {\n      \"generated_by\": \"large-payload-tester workflow\",\n      \"repository\": \"PLACEHOLDER_REPO\",\n      \"workflow_run_url\": \"PLACEHOLDER_URL\"\n    }\n  },\n  \"padding\": \"\"\n}\nEOF\n\n# Use jq to properly populate the JSON with dynamic values and generate large array\n# Generating 2000 items + 400KB padding to create ~500KB file\n# secret_reference is at the top level for easy extraction after large payload retrieval\njq --arg secret \"$TEST_SECRET\" \\\n   --arg run_id \"$EXPR_GITHUB_RUN_ID\" \\\n   --arg timestamp \"$(date -Iseconds)\" \\\n   --arg repo \"$EXPR_GITHUB_REPOSITORY\" \\\n   --arg url \"$EXPR_GITHUB_SERVER_URL/$EXPR_GITHUB_REPOSITORY/actions/runs/$EXPR_GITHUB_RUN_ID\" \\\n   '.secret_reference = $secret |\n    .test_run_id = $run_id | \n    .test_timestamp = $timestamp | \n    .data.metadata.repository = $repo | \n    .data.metadata.workflow_run_url = $url | \n    .data.large_array = [range(2000) | . as $i | {id: $i, value: (\"item-\" + tostring), random_data: (\"rand-\" + ($i * 17 % 9973 | tostring) + \"-\" + ($i * 31 % 8191 | tostring)), extra_data: (\"data-\" + tostring + \"-\" * 50)}] |\n    .padding = (\"X\" * 400000)' \\\n   $TEST_FS/$LARGE_PAYLOAD_FILE > $TEST_FS/$LARGE_PAYLOAD_FILE.tmp\n\nmv $TEST_FS/$LARGE_PAYLOAD_FILE.tmp $TEST_FS/$LARGE_PAYLOAD_FILE\n\n# Verify file was created and is large enough\nFILE_SIZE=$(wc -c < $TEST_FS/$LARGE_PAYLOAD_FILE)\necho \"Created $LARGE_PAYLOAD_FILE with size: $FILE_SIZE bytes (~$(($FILE_SIZE / 1024))KB)\"\nif [ \"$FILE_SIZE\" -lt 512000 ]; then\n  echo \"WARNING: Test file is smaller than expected ($FILE_SIZE bytes < 500KB)\"\n  echo \"Continuing with test anyway...\"\nfi\n\necho \"Test environment setup complete\"\necho \"Large file stored in: $TEST_FS/$LARGE_PAYLOAD_FILE\"\ngrep -H $TEST_SECRET $TEST_FS/$LARGE_PAYLOAD_FILE\necho \"Secret stored in $TEST_FS/$SECRET_FILE\"\ngrep -H $TEST_SECRET $TEST_FS/$SECRET_FILE\n"


+1. Use the **`filesystem-read_file` MCP tool** to read `/workspace/large-test-file.json`.
+2. The file is ~500KB so the gateway will return a truncated response containing a `payloadPath` field.
+3. Use bash to read the full JSON from `payloadPath` (e.g. `cat <payloadPath>`).
+4. Extract the top-level `secret_reference` field from that JSON.
+5. Use the **`filesystem-read_file` MCP tool** to read `/workspace/secret.txt`.


          .data.metadata.workflow_run_url = $url | 
-          .data.large_array = [range(2000) | . as $i | if $i == 1000 then {id: $i, value: ("item-" + tostring), secret_reference: $secret, extra_data: ("data-" + tostring + "-" * 50)} else {id: $i, value: ("item-" + tostring), random_data: ("rand-" + ($i * 17 % 9973 | tostring) + "-" + ($i * 31 % 8191 | tostring)), extra_data: ("data-" + tostring + "-" * 50)} end] |
+          .data.large_array = [range(2000) | . as $i | {id: $i, value: ("item-" + tostring), random_data: ("rand-" + ($i * 17 % 9973 | tostring) + "-" + ($i * 31 % 8191 | tostring)), extra_data: ("data-" + tostring + "-" * 50)}] |
          .padding = ("X" * 400000)' \


lpcox · 2026-06-17T15:05:22Z

@copilot address the review feedback

Initial plan

1ecb262

Copilot AI assigned Copilot and lpcox Jun 17, 2026

Copilot started work on behalf of lpcox June 17, 2026 14:06 View session

GitHub Advanced Security started work on behalf of lpcox June 17, 2026 14:06 View session

GitHub Advanced Security finished work on behalf of lpcox June 17, 2026 14:08

Fix Large Payload Tester: move secret to top level and clarify agent …

be142d8

…prompt

GitHub Advanced Security started work on behalf of lpcox June 17, 2026 14:18 View session

Copilot AI changed the title ~~[WIP] Fix large payload tester engine failure~~ Fix Large Payload Tester: move secret to top level, harden agent prompt Jun 17, 2026

Copilot finished work on behalf of lpcox June 17, 2026 14:19

Copilot AI requested a review from lpcox June 17, 2026 14:19

GitHub Advanced Security finished work on behalf of lpcox June 17, 2026 14:20

lpcox marked this pull request as ready for review June 17, 2026 14:52

Copilot AI review requested due to automatic review settings June 17, 2026 14:52

Copilot started reviewing on behalf of lpcox June 17, 2026 14:52 View session

Copilot AI reviewed Jun 17, 2026

View reviewed changes

Copilot started work on behalf of lpcox June 17, 2026 15:05 View session

lpcox merged commit 610f57a into main Jun 17, 2026
40 of 41 checks passed

lpcox deleted the copilot/fix-large-payload-tester-issue branch June 17, 2026 15:07

Copilot stopped work on behalf of lpcox due to an error June 17, 2026 15:07
The session was cancelled by the user.

lpcox restored the copilot/fix-large-payload-tester-issue branch June 17, 2026 15:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix Large Payload Tester: move secret to top level, harden agent prompt#7681

Fix Large Payload Tester: move secret to top level, harden agent prompt#7681
lpcox merged 2 commits into
mainfrom
copilot/fix-large-payload-tester-issue

Copilot AI commented Jun 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

lpcox commented Jun 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Copilot AI commented Jun 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

lpcox commented Jun 17, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented Jun 17, 2026 •

edited

Loading