forked from lucas-maes/le-wm
-
Notifications
You must be signed in to change notification settings - Fork 1
harness: ingest LLM candidate packs into the hard benchmark #421
Copy link
Copy link
Closed
Labels
area:evaluationArea: evaluationArea: evaluationarea:harnessArea: harnessArea: harnessarea:securityArea: securityArea: securityeffort:mMedium implementation changeMedium implementation changepriority:p1Required for v1.0 or core follow-throughRequired for v1.0 or core follow-throughspec:rfc-0016Derived from RFC-0016Derived from RFC-0016type:featureFeature implementation workFeature implementation work
Description
Metadata
Metadata
Assignees
Labels
area:evaluationArea: evaluationArea: evaluationarea:harnessArea: harnessArea: harnessarea:securityArea: securityArea: securityeffort:mMedium implementation changeMedium implementation changepriority:p1Required for v1.0 or core follow-throughRequired for v1.0 or core follow-throughspec:rfc-0016Derived from RFC-0016Derived from RFC-0016type:featureFeature implementation workFeature implementation work
Parent
#417
What to build
Allow the hard downstream benchmark to consume manifest-backed LLM candidate packs while preserving the existing OpenRouter/BYOK, redaction, non-execution, and secret-scan boundaries.
Acceptance criteria
codelewm.llm_candidate_pack.v1artifacts as candidate sources.Blocked by
#419