Skip to content

Add seller-copilot: a consultant front door for broad and multi-part seller questions - #27

Open
fiharet34 wants to merge 3 commits into
joomcode:mainfrom
fiharet34:add-seller-copilot-skill
Open

Add seller-copilot: a consultant front door for broad and multi-part seller questions#27
fiharet34 wants to merge 3 commits into
joomcode:mainfrom
fiharet34:add-seller-copilot-skill

Conversation

@fiharet34

Copy link
Copy Markdown

Summary

Adds one skill, seller-copilot, with 15 references/ files.

The catalogue's 17 existing skills each do one job well. This one covers the two request
shapes they don't:

  1. Vague or multi-part questions — "me ajuda a crescer", "por onde começo", "analisa
    minha operação" — where the seller doesn't yet know which analysis they need. Today these
    have no entry point.
  2. Deeper analyses with no focused twin here — real margin and Mercado Livre fees,
    pricing and price wars, whether a product is worth selling, finding new products, vetting
    a supplier catalogue, import-vs-domestic, diagnosing a listing that doesn't sell,
    parameter-level competitor benchmarking, assortment and price gaps, market structure, and
    seasonality.

It classifies the request, asks at most one or two clarifying questions, runs the minimal set
of procedures from references/, and returns one consolidated verdict-first answer rather
than a pile of tables.

It routes to your skills rather than competing with them

This is deliberate and load-bearing:

  • SKILL.md opens with "When to use another skill instead", naming eight existing skills
  • Every routing table hands off by name where one already does the job — a single category
    snapshot to category-opportunity-index, buy-box comparison to my-product-vs-catalog,
    growth ranking to growing-leaf-category-tracker, trending terms to
    top-keywords-in-my-category
  • Three reference files (category-evaluation, keyword-intel, competitor-profile) point
    at their focused twin first and explain when the reference procedure fits better instead
  • The frontmatter description deliberately omits trigger phrases for capabilities that
    already have a focused skill, so it doesn't intercept their traffic

Verified in testing: of nine single-purpose questions, this skill correctly stayed out of the
way on four and the focused skill took them.

Structure

skills/seller-copilot/
├── SKILL.md              (293 lines; description 983/1024 chars)
└── references/           (15 files, ~1,200 lines)

Only the frontmatter sits in the always-loaded metadata budget. The body loads on activation;
a reference file is read only when that specific analysis runs.

This is the first skill here to use a references/ subdirectory, and CONTRIBUTING.md
describes one folder containing a single SKILL.md. Flagging it as the main thing to accept or
reject. The reason for the deviation: bundling 15 analyses costs one metadata entry and one
README row instead of 15, and 15 separate skills couldn't reference each other once each is
zipped individually for the desktop install path. If you'd rather have the flat shape,
splitting is mechanical — no external contract depends on the layout.

Evidence

Run through a real agent CLI against the live JoomPulse connector, scored by an LLM judge plus
deterministic disclosure checks, against two comparators on identical questions and criteria:
the internal predecessor this skill is derived from, and a bare prompted agent.

Nine multi-part scenarios:

bare agent internal predecessor this skill
avg judge (1–5) 3.61 4.06 4.39
disclosure checks passed 6/9 7/9

Eight further scenarios written for the procedures the existing criteria never exercise
averaged 4.00 — or 4.67 excluding two items that measure something other than this
skill (below).

Across both rounds the scenarios surfaced eight real defects in these procedures — provenance
mislabels in both directions, a ranking padded past the data it returned, dates recalled from
the model's own knowledge presented as marketplace data, and a fallback that could not produce
the output its own rule demanded. All eight are fixed here. For the six with a measurable output
signature, the affected scenario was re-run to confirm the behaviour changed and not merely
the score; the remaining two are structural and were not separately re-measured. The commit
messages carry the detail.

The honest summary of the whole exercise: answer quality was close to the internal
predecessor's, and the repeatable difference was disclosure discipline
— labelling estimates
as estimates, naming what data is unavailable instead of filling the gap, and carrying caveats
that are easy to drop. So that's what's made structural here: every reference file ends with a
mandatory Boundaries section, the estimate disclaimer is required once per answer, is
required for missing values, and "never fabricate a number" is a hard rule.

What this evidence does not prove

  • Two of the eight scenarios don't measure this skill. One scored 1/5 because the question
    routed to an existing focused skill lacking a caveat this skill's references require —
    routing is decided by frontmatter matching before this skill loads, so that score reflects
    where the question went rather than anything in these files. The other ran under a
    harness flag telling the agent no user was available to answer, which contradicts this skill's
    ask-first rule; interactively the same question behaves as intended.
  • The strict tool-evidence check fails on every item and is excluded as inapplicable — the
    existing criteria expect the predecessor's tool and data-model naming.
  • Three capabilities are unmeasured: supplier vetting against a real supplier list, listing
    diagnosis end-to-end, and keyword intelligence.
  • Samples are small (9 and 8 items). Every score was read against the answer text rather than
    taken at face value — that's how two of the six defects above were told apart from checker
    artifacts.

Notes for reviewers

  • Pure addition. No existing skill, asset or row is modified. The only change to an
    existing file is one appended README table row.
  • Works standalone. Where a step names another skill, SKILL.md says to use it if
    installed and otherwise perform the equivalent analysis directly — so the skill works on its
    own via per-skill .zip install as well as alongside the full catalogue.
  • Mercado Livre (Brasil) only, read-only, no real-time claims.
  • Three commits, kept unsquashed on purpose: the initial skill, the six evaluation-driven
    fixes, and the handoff-caveat change. The fix history is the part that shows the skill was
    measured rather than just written.

If anything here isn't what you want, say so and I'll change it — the references/ layout,
the scope of the description, which requests route here versus to an existing skill, or any
wording in the reference files. Splitting it into flat skills, trimming it, or narrowing what it
claims are all straightforward, and I'd rather adjust to fit the catalogue's conventions than
have it land as an exception. Same goes for anything the review turns up that needs following up
elsewhere.

Public-Safety Checklist

  • No credentials, tokens, keys, or secrets.
  • No internal filesystem paths.
  • No private Slack, Jira, Notion, admin, or service URLs.
  • No private service names or implementation details.
  • No customer data or private product data.
  • Skill directory name matches SKILL.md frontmatter name.

scripts/check-public-safety.sh passes. No data-model or field names, tool or resource
identifiers, internal service names, paths, or customer data appear anywhere in these files —
they describe procedures and judgement only.

fiharet34 and others added 3 commits July 29, 2026 18:08
A consultant front door for the two request shapes the existing 17 focused
skills do not cover: vague or multi-part questions ("me ajuda a crescer"),
and the deeper analyses with no focused twin here — real margin and fee
breakdowns, pricing strategy, product validation, supplier vetting,
import-vs-domestic, listing diagnosis, competitor benchmarking, assortment
and price gaps, market structure, and seasonality.

One skill directory with 15 references/ files, using the progressive-disclosure
model: only the frontmatter sits in the always-loaded metadata budget, the body
loads on activation, and a reference file is read only when that analysis runs.
This is the first skill here to use a references/ subdirectory — the existing
ones are all a single flat SKILL.md.

Routes to the existing skills rather than competing with them. SKILL.md opens
with a "When to use another skill instead" section naming eight of them, every
routing table hands off by name where one already does the job, and the
frontmatter description deliberately omits trigger phrases for capabilities
that already have a focused skill.

Evaluated against a JoomPulse MCP baseline through a real agent CLI. On six
multi-part scenarios it averaged 4.33/5 on an LLM judge versus 4.00 for the
internal predecessor and 3.42 for a bare prompted agent, and passed the
required-disclosure check on 6/6 versus 5/6. Two real defects that testing
surfaced are fixed here: external dates used as table annotations now carry
a source marker, and cross-category rankings must be grounded row-by-row in
returned data with the composite scoring shown rather than padded to a
requested count.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six fixes, each traced to a specific failure in eight real runs against the
live connector rather than to review taste.

Provenance, two live mislabels:
- competitor-profile.md declared the whole profile estimated, but a seller's
  completed-sales counter, sales trend, reputation, medal and cancellation
  rate are real marketplace history. Calling a real figure an estimate pushes
  the seller to discount the most solid evidence in the profile. Provenance is
  now marked per figure, and SKILL.md's Scope names the real-history set.
- An answer labelled average ticket "real market data". It is estimated
  revenue over estimated sales, and "ticket" reads like "price", which is
  real. SKILL.md now states that a ratio derived from estimates is itself an
  estimate.

benchmark.md demanded a losing/behind/winning scorecard unconditionally, which
the public fallback cannot produce — there is no measured seller value to band
— so a run improvised a market report and lost the analysis's point. The
fallback now keeps the scorecard shape with conditional bands the seller can
place themselves in.

Step 2 had a real rule collision: Step 1 says do not guess when the request is
too vague, Step 2 says never block and proceed with defaults, and no default
was defined for the intent itself. Resolved explicitly, without weakening the
ask-first path that works when the seller can answer.

Routing gaps: the body advertised "analisa minha operação" with no row for the
seller's own store, and had no row for "why did my sales drop" — whose
load-bearing judgement (a seasonal dip is not a decline) already existed but
was unreachable.

find-new-products.md now says a stock budget constrains unit cost, which the
marketplace does not show, so selling price is only an upper-bound proxy.

The keyword handoff now carries the supply-depth caveat inline, so the caveat
survives on traffic this skill routes. Note it cannot survive traffic routed
by frontmatter matching before this skill loads — that belongs to the skill
rendering the figure, and is reported to the maintainers separately rather
than worked around here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deferring to a focused skill is usually right, but a focused skill states only
the caveats its own job needs — so a caveat this skill's references make
mandatory can silently disappear the moment work is handed off.

Evaluation caught this concretely on keyword work: a per-term product count
counts sellers targeting a term, not shoppers searching it, and an answer
presented it as "most searched" with no correction. The same hazard applies to
growth rankings, where dropping tiny-absolute-revenue niches before ranking is
what separates a useful ranking from noise.

Both now live in one "Caveats that must survive a handoff" section with the
general rule, rather than crammed into table cells, so the list can grow
without the routing tables becoming unreadable.

Note the limit of what this can achieve: it governs handoffs this skill makes.
Where a question routes to a focused skill by frontmatter matching before this
skill ever loads, no text here can intercept it — that belongs to the skill
rendering the figure, and is reported to the maintainers separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@fiharet34
fiharet34 marked this pull request as ready for review July 31, 2026 14:27
@fiharet34
fiharet34 requested a review from profunctor as a code owner July 31, 2026 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant