docs: allocation efficiency — the metric, its guards, and the 103-run baseline (CG-9)
Records the sweep over every A/B log on this machine (103 sessions, 297 explore calls, 0 crashes) and, more usefully, the new-vs-baseline arm table the metric exists for: express 82% → 100%, cg21/client-go 67% → 95%, two pairs going the other way. States the caveat in the places it can be misread: the corpus median sits in the eighties because these are flow questions whose answers name most of the chain, the metric is byte-weighted, and an agent can use a file without citing it. It compares two builds on one question; it is not an absolute waste figure.
This commit is contained in:
@@ -51,6 +51,11 @@ rendered markdown of the responses the agent actually received. The CG-4 diagnos
|
||||
only exists on the new build, so it cannot measure the baseline arm; the markdown parse is the
|
||||
only instrument that measures both arms the same way.
|
||||
|
||||
That `--answer <glob>` is hand-specified ground truth. Every run now also reports the same
|
||||
intersection inferred from the agent's own final answer, so it needs no per-question setup —
|
||||
see [`explore-allocation-efficiency.md`](explore-allocation-efficiency.md), which re-scores
|
||||
the arms below (express: 82% baseline → 100% new).
|
||||
|
||||
| Repo | Lang | Files | Generated | Tier | Role |
|
||||
|---|---|---|---|---|---|
|
||||
| `kubernetes/client-go` | Go | 2,454 | 2,001 | medium (2 calls / 28K) | **the #1500 shape** — generated CRUD beside hand-written machinery |
|
||||
|
||||
Reference in New Issue
Block a user