Bug
When a code-grader script returns assertions but omits score, the score defaults to 0 — even if all assertions pass:
// packages/core/src/evaluation/graders/code-grader.ts
const score = clampScore(typeof parsed?.score === 'number' ? parsed.score : 0);
So {"assertions": [{"text": "ok", "passed": true}]} → score 0. Wrong.
Expected Behaviour
score present → use it (explicit override for weighted/non-binary scoring)
score absent + assertions present → derive as passing / total
- Neither present → score 0
Impact
Scripts currently must redundantly compute score themselves to avoid getting 0:
passed = sum(1 for a in assertions if a["passed"])
score = passed / len(assertions) if assertions else 0.0
print(json.dumps({"score": score, "assertions": assertions}))
With the fix, the score line becomes optional boilerplate — scripts can just return assertions and the framework computes the score.
Bug
When a
code-graderscript returnsassertionsbut omitsscore, the score defaults to 0 — even if all assertions pass:So
{"assertions": [{"text": "ok", "passed": true}]}→ score 0. Wrong.Expected Behaviour
scorepresent → use it (explicit override for weighted/non-binary scoring)scoreabsent +assertionspresent → derive aspassing / totalImpact
Scripts currently must redundantly compute
scorethemselves to avoid getting 0:With the fix, the
scoreline becomes optional boilerplate — scripts can just return assertions and the framework computes the score.