Add prefetch_strategy A/B experiment to weekly-blog-post-writer

[Copilot]

This updates weekly-blog-post-writer to run a controlled lazy-vs-eager prefetch experiment for release/PR data collection. The goal is to measure whether moving initial GitHub fetches into setup steps reduces agent turns and run duration without hurting PR creation outcomes.

• Experiment config (frontmatter)

  • Added experiments.prefetch_strategy in rich object form with:
    • variants: [lazy, eager]
    • primary metric agent_turn_count
    • secondary metrics run_duration_ms, pr_creation_rate
    • guardrail pr_creation_success_rate threshold 0.80
    • min_samples: 10, weight: [50, 50], start_date, and tracking issue metadata.

• Eager variant prefetch step

  • Added a conditional steps entry gated by selected experiment variant.
  • In eager, the workflow pre-fetches and stores:
    • /tmp/gh-aw/agent/releases.json
    • /tmp/gh-aw/agent/merged-prs.json

• Prompt behavior split (Step 2/3)

  • Step 2 and Step 3 now branch on experiments.prefetch_strategy:
    • eager: read pre-fetched JSON files (no runtime list_releases / list_pull_requests call).
    • lazy: preserve current tool-driven behavior.

• Compiled workflow output

  • Regenerated weekly-blog-post-writer.lock.yml to reflect experiment activation outputs and conditional prefetch wiring.

experiments:
  prefetch_strategy:
    variants: [lazy, eager]
    metric: agent_turn_count
    secondary_metrics: [run_duration_ms, pr_creation_rate]
    guardrail_metrics:
      - name: pr_creation_success_rate
        direction: min
        threshold: 0.80

steps:
  - name: Pre-fetch release and merged PR data
    if: ${{ needs.activation.outputs.prefetch_strategy == 'eager' }}
    run: |
      # writes /tmp/gh-aw/agent/releases.json and merged-prs.json

[github-actions]

Hey @Copilot 👋 — thanks for kicking off the A/B prefetch-strategy experiment for the weekly blog post writer! The direction looks solid. A couple of things to address before this is ready for review:

• Add a PR description — the body right now is the agentic task checklist (the how), but reviewers need a short summary of what is changing and why — e.g. what problem the experiments.prefetch_strategy block solves, what the eager vs lazy strategies do differently, and what outcome is expected.
• No files changed yet — the PR is still at the "Initial plan" commit with 0 lines changed. The checklist items have not been executed yet. Once the implementation lands, make sure test coverage for the new prefetch-strategy path is included.

Once the implementation is complete, here is a prompt you can use to finish this off cleanly:

This is PR #38622 "[WIP] Add A/B test for prefetch strategy in weekly blog post writer".

1. Replace the current PR body (which is just the agentic task checklist) with a proper PR description that explains:
   - What change is being made (adding an `experiments.prefetch_strategy` frontmatter block to `.github/workflows/weekly-blog-post-writer.md` with eager vs lazy prefetch steps)
   - Why it is needed (the motivation / problem being solved)
   - What the expected outcome or observable difference is
2. Ensure the implementation includes tests or validation steps that exercise the new prefetch-strategy code path.
3. Remove the `[WIP]` prefix from the title and mark the PR as ready for review.

Generated by ✅ Contribution Check · 560.3 AIC · ⌖ 13.4 AIC · ⊞ 24.7K · ◷

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds an A/B experiment to the “Weekly Blog Post Writer” agent workflow to evaluate whether eagerly prefetching GitHub releases and merged PRs reduces agent tool calls and overall runtime.

Changes:

• Introduces an experiments.prefetch_strategy (lazy vs eager) spec with metrics/guardrails.
• Adds an eager-only prefetch step to write releases + merged PRs JSON to /tmp/gh-aw/agent/.
• Updates the prompt instructions to consume the prefetched JSON when the eager variant is selected.

Show a summary per file

File	Description
.github/workflows/weekly-blog-post-writer.md	Adds experiment config, eager prefetch step, and prompt branching to read prefetched data.
.github/workflows/weekly-blog-post-writer.lock.yml	Propagates experiment selection through the locked workflow and implements prefetch/proxy + experiment state persistence.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 2/2 changed files
• Comments generated: 5