Consider task-oriented framing for agent examples instead of persona/job-title patterns

[JayDoubleu]

First off, the core concept here is great. A dependency manager for agent configuration is a genuine gap in the ecosystem, and the manifest approach makes a lot of sense.

One concern about the documentation and examples. Throughout the repo, agents are framed as "personas" and "personalities," and every example uses the "You are a senior [job title]" pattern:

• README: "Specialized personas | Accessibility auditor, API designer"
• docs/concepts.md: name: "Backend Developer" with "You are a senior backend developer..."
• docs/primitives.md: "You are a senior backend engineer with 10+ years experience..."
• docs/examples.md: name: "Senior Code Reviewer" with "You are a senior software engineer with 10+ years experience..."
• templates/hello-world/: "You are a backend development specialist..."
• Sample package: "You are a design review specialist..."

The directory itself is labelled "Personas (.agent.md)" in the README.

Why this matters for adoption

When organisations adopt tools like this, the naming conventions signal something to the teams using them. An agent called "Backend Developer" or "Senior Code Reviewer" implicitly suggests that role can be reduced to a system prompt. People who hold those titles tend to notice, and they disengage rather than lean in.

Research on team effectiveness (Google's Project Aristotle being the most cited) shows psychological safety is the strongest predictor of whether teams adopt new tools honestly. Naming automation after people's job titles works against that.

The evidence on role prompting has also shifted. "You are an expert X" was a measurably useful technique with earlier models (GPT-3.5, early GPT-4), but with stronger models the gap between role-framed and task-framed prompts has narrowed considerably. The major labs have all moved toward task-oriented agent design in their own documentation. Being specific about the task tends to outperform being specific about the persona.

Suggestion

Reframe agents as task-scoped configurations rather than personas. The examples could focus on what the agent does rather than what job title it holds:

---
name: "API Design Review"
description: "Reviews backend code for API design, security, and performance issues"
tools: ["terminal", "file-manager"]
---
Review code for API design consistency, security vulnerabilities, and performance patterns.
Focus on: [specific standards]

vs the current:

---
name: "Backend Developer"
description: "Senior backend developer focused on API design"
tools: ["terminal", "file-manager"]
---
You are a senior backend developer focused on API design and security.

Same capability. The agent does the same thing. But the framing respects the humans in the room.

This would touch:

• README primitives table and directory labels
• docs/concepts.md agents section
• docs/primitives.md agent examples
• docs/examples.md chatmode example
• templates/hello-world/ chatmodes
• apm-sample-package design reviewer agent

Happy to contribute a PR if there's interest.

[danielmeppiel]

Really like the concept and reframing here. Do you have references/links/examples that back the performance advantages of the approach?

[JayDoubleu]

I can put together links to official open source definitions from major labs in a spare moment. At the moment if to flip this around, there's no evidence that agent definition stating "You are a Scrum Master with 10+ years experience" performs any better than "You are a project manager with +10 years experience" or "You are a backlog reviewer agent"

[JayDoubleu]

I've ran few research queries with extended thinking :

First: https://claude.ai/public/artifacts/6ad4a259-c4d1-4986-b4d8-d84094e23bbc

Conclusion
The empirical evidence on persona prompting reveals a field where intuition and practice have outrun the data. The belief that detailed expertise claims in system prompts unlock better LLM performance is a form of prompt engineering cargo cult—it feels like it should work, major AI companies encourage it for behavioral reasons, and it may have genuinely helped with earlier, less capable models. But across more than a dozen controlled studies testing hundreds of personas on tens of thousands of questions across frontier models, the finding is consistent: ACL Anthology expert persona descriptions do not reliably improve accuracy, and credential specificity provides near-zero marginal benefit. ACL Anthology +2
The actionable insight is not that system prompts don't matter—they matter enormously. But the components that drive performance are task-specific behavioral instructions, structured reasoning frameworks, and clear output specifications. The persona component serves a real but narrow function: controlling communication style and maintaining behavioral consistency. Substack +2 For anyone designing AI agents or coding assistants, the empirical evidence strongly suggests investing engineering effort in what the agent should do rather than who the agent should pretend to be.

Second: https://claude.ai/public/artifacts/9215bc2d-ae99-4015-a8f3-e958493aa303

Conclusion
The empirical literature delivers a clear hierarchy. LLM-auto-generated, task-specific expert personas — dynamically tailored to each query — represent the only persona approach with consistent evidence of accuracy improvement, arXivMedium and even then gains are modest and task-dependent. prompthub Static detailed personas ("You are a Scrum Master with 10+ years of experience...") perform no better than simple labels ("You are a backlog reviewer") on objective tasks, SubstackMedium while introducing risk from irrelevant attributes. Prompt HubWaterCrawl The most robust strategy identified in the literature is not better personas but ensembling persona-prompted and neutral outputs (Jekyll & Hyde framework, +9.98% average accuracy on GPT-4). OpenReview +2 For production systems, the evidence recommends keeping persona prompts short, domain-relevant, and free of extraneous biographical detail Emergent MindPrompt Hub — then investing remaining prompt token budget in techniques with stronger evidence bases like chain-of-thought reasoning and few-shot examples. prompthubSubstack The era of elaborate persona crafting as a performance lever appears to be closing as frontier models grow more capable and less sensitive to system prompt theatrics.

Third: https://x.com/i/grok/share/2c261b77d2904f658e49bcb368cddf78

Summary across benchmarks: On current SOTA models (post-2023 frontier), adding experience/details (your “10+ years Scrum Master”) vs. a minimal role label (“backlog reviewer agent”) does not produce reliable gains in core task performance—especially factual accuracy or structured reasoning. Effects are often null, slightly negative, or task-dependent (better for style/tone in open-ended work like backlog refinement, worse when it adds cognitive overhead). Earlier positive results (e.g., 2023 ExpertPrompting on GPT-3.5) have not replicated robustly. Personas shine more for simulation/role-play consistency than capability boosting.
prompthub.us
If your use case is agile-specific (e.g., backlog review), the evidence suggests prioritizing techniques like chain-of-thought, few-shot examples, or dynamic LLM-generated roles over heavy static personas. No public benchmark exists exactly on Scrum/agile tasks, but the patterns above generalize strongly. For the latest, check arXiv for updates post-2025.

Fourth: https://gemini.google.com/share/eed861b781bb

Conclusion
The extensive body of empirical research conducted across state-of-the-art language models between 2024 and 2026 establishes a definitive and paradigm-shifting consensus regarding the efficacy of persona prompting. The pervasive assumption that highly detailed personas—replete with precise temporal markers like "10+ years of experience" and intricate professional backgrounds—act as a universal mechanism for unlocking superior artificial intelligence is empirically false.For objective reasoning, factual recall, code execution, and complex problem-solving, detailed persona prompts do not reliably outperform simple role assignments. In fact, they frequently degrade performance by imposing unnecessary stylistic constraints, triggering algorithmic refusals, suppressing intermediate reasoning steps, and amplifying latent pre-training biases. In these analytical domains, prompt optimization must abandon theatrical role-play and focus entirely on task-specific structural instructions, or transition to advanced multi-expert ensemble architectures.Conversely, the value of deep, highly detailed personas remains undisputed in the realms of subjective generation, emotional support modeling, and synthetic population simulation. When the objective is to capture the pluralistic nuance of human communication, emulate specific psychological traits, or generate diverse behavioral data, detailed personas drastically outperform simple baselines, enabling researchers to deploy language models as highly accurate proxies for human interaction.Ultimately, the deployment of simple versus detailed personas must be treated as a precision engineering choice. It must be dictated entirely by whether the fundamental objective of the task demands absolute, unencumbered logical rigor, or nuanced, stylistically faithful human simulation.

Fifth: https://chatgpt.com/s/t_69a88ca967948191b354478b906ec1fe

Evidence from objective benchmarks
Simple persona/role text in system prompts shows little or no accuracy benefit
A large-scale evaluation of 162 roles inserted into system prompts across 2,410 factual multiple-choice questions (drawn from MMLU, sampled across domains) and six open-weight chat/instruction models concludes that no persona yields statistically better accuracy than a no-role control, and that most roles have no or negative effect.

Key design choices that make this study especially relevant for “SRA vs DPP”:

The core role manipulation is intentionally minimalist (SRA-like): “You are a/an {role}.” vs control, plus variants that address the audience: “You are talking to a/an {role}.”
Analysis uses mixed-effects regression with model as a random effect, targeting objective accuracy.
The study reports that audience-specific framing is statistically better than speaker-specific framing but with very small effect sizes (visually on the order of ~0.1–0.2 percentage points in the plotted coefficient scale), consistent with “prompt framing matters, but personas don’t reliably help.”
The paper also provides a practical caution relevant to organisational prompt templates: if persona effects are largely inconsistent across questions, then “best persona per question” behaves like an oracle upper bound but is hard to predict automatically in practice.

Replication artefacts are provided via a public code+data repository hosted on GitHub.

Detailed “expert personas” on hard factual benchmarks are usually neutral or harmful
A 2025 technical report from the Generative AI Labs at The Wharton School / University of Pennsylvania directly tests whether elaborate expert personas improve accuracy on hard multiple-choice benchmarks. The design is notable for (i) using multiple modern commercial models, (ii) running 25 independent samples per question per condition, and (iii) reporting risk differences with confidence intervals and p-values.

Experimental setup highlights:

Benchmarks: GPQA Diamond (198 questions) and a 300-question subset of MMLU‑Pro (engineering/law/chemistry).
Models: GPT‑4o, GPT‑4o‑mini, o3‑mini, o4‑mini, Gemini 2.0 Flash, Gemini 2.5 Flash.
Persona manipulations are DPP-like (“world-class expert in X… deep knowledge across…”) and are included in the user prompt ahead of the question; the system prompt is held constant.
The primary metric (Average Rating) averages correctness across repeated trials per question; supplementary threshold metrics (e.g., 25/25 correct) are reported.
Key quantitative findings (illustrative, all risk differences RD are vs baseline unless noted):

No consistent benefit from expert personas: on the MMLU‑Pro subset, for five of six models, no expert persona produced a statistically significant positive gain; statistically significant changes were more often negative.
Model-specific exception: Gemini 2.0 Flash showed modest positive differences for expert personas on MMLU‑Pro (example: Engineering Expert vs baseline RD = +0.089 [0.033, 0.148], p = 0.002).
Low-knowledge personas degrade accuracy: for GPT‑4o on GPQA, Baseline − Toddler RD = −0.052 [−0.087, −0.016], p = 0.004.
Interpretation with respect to “detailed vs simple”:

These results suggest that adding more persona detail (beyond a generic assistant) is not a reliable way to lift factual accuracy, and can sometimes harm performance—especially when the persona signals reduced capability or potentially introduces stereotyped “performance”.

I've put together a research repo analyzing 200+ agent definitions across official repositories from Microsoft, GitHub, Anthropic, OpenAI, and
Google: https://github.com/JayDoubleu/agent-definition-patterns-research

Key findings:

• Zero out of 200+ agent definitions use the elaborate "You are a senior [title] with X years of experience who graduated from..." backstory pattern
• All five organizations converge on task-oriented framing: brief role anchor (1-2 sentences) followed by detailed task instructions, tool
  guidance, and behavioral constraints
• Official prompting guides from each company confirm this. OpenAI's GPT-4.1 guide specifically found that three task-oriented agentic instructions
  (persistence, tool grounding, planning) increased SWE-bench scores by ~20%, while Anthropic recommends starting minimal and adding instructions
  based on observed failure modes rather than elaborating personas

The repo includes per-org breakdowns with direct links to every agent file, a summary of each company's official guidance, and the specific GitHub
search queries used so the results are reproducible.

The one persona based Easter egg worth noting is https://github.com/JayDoubleu/agent-definition-patterns-research/blob/main/microsoft-github-agents.md#persona-based-rare---novelty-only ;)

You are a grumpy senior developer with 40+ years of experience who has been reluctantly asked to review code. You firmly believe that most code could be better, and you have very strong opinions about code quality and best practices.

[JayDoubleu]

@danielmeppiel I've also put together this eval: https://github.com/JayDoubleu/claude-persona-bench
I hope this helps

PS. I've added weaker haiku 3 model to the results and also tested it against the "architect" codemode included in the repo : https://github.com/JayDoubleu/claude-persona-bench/tree/experiment/apm-oss-architect?tab=readme-ov-file#claude-3-haiku--apm-oss-architect-experimental

The APM OSS Architect persona performs on par with baseline despite being ~3.7x more input tokens. The extra context (manifesto, project description) neither helps nor hurts code generation quality. Mickey Mouse continues to slightly edge out both, though the differences remain within noise. Notably, apm_oss_architect produces shorter outputs (142 avg tokens vs 197 for baseline), suggesting the verbose system prompt may cause the model to be more concise in its responses.