docs: allocation efficiency — the metric, its guards, and the 103-run baseline (CG-9)

Records the sweep over every A/B log on this machine (103 sessions, 297
explore calls, 0 crashes) and, more usefully, the new-vs-baseline arm
table the metric exists for: express 82% → 100%, cg21/client-go 67% →
95%, two pairs going the other way.

States the caveat in the places it can be misread: the corpus median sits
in the eighties because these are flow questions whose answers name most
of the chain, the metric is byte-weighted, and an agent can use a file
without citing it. It compares two builds on one question; it is not an
absolute waste figure.
This commit is contained in:
Colby McHenry
2026-08-05 00:44:30 -05:00
parent db3b8d2a1e
commit fa15d1046a
2 changed files with 200 additions and 0 deletions
@@ -51,6 +51,11 @@ rendered markdown of the responses the agent actually received. The CG-4 diagnos
only exists on the new build, so it cannot measure the baseline arm; the markdown parse is the
only instrument that measures both arms the same way.
That `--answer <glob>` is hand-specified ground truth. Every run now also reports the same
intersection inferred from the agent's own final answer, so it needs no per-question setup —
see [`explore-allocation-efficiency.md`](explore-allocation-efficiency.md), which re-scores
the arms below (express: 82% baseline → 100% new).
| Repo | Lang | Files | Generated | Tier | Role |
|---|---|---|---|---|---|
| `kubernetes/client-go` | Go | 2,454 | 2,001 | medium (2 calls / 28K) | **the #1500 shape** — generated CRUD beside hand-written machinery |