Two permanent fixtures pinning the failure mode from issue #1500 — explore
spending its byte envelope on files that merely name-collide with the query.
BOTH FAIL TODAY, by design: they document the bug and become the pass gate
for CG-10 (scoring) + CG-12 (proportional allocation).
__tests__/fixtures/payroll-go/ — a synthetic Go service mirroring the
reporter's shape: generated FKIT CRUD beside a hand-written payroll use-case,
entered from an HTTP route. Half the generated tree carries ORDINARY names
detectable only by their `// Code generated ... DO NOT EDIT.` header (the
#1500 case, and end-to-end cover for CG-5); `payrollpb/*.pb.go` covers the
path-detectable channel. BuildPayslip, Upsert and Store each exist twice,
generated and hand-written. cycle.go sits above the whole-file window so it
clips; the generated files sit below it so they ship whole.
Asking "how does payroll cycle create and calculate payslips?" — naming none
of the answering symbols — the generated CRUD delivers 57.4% of the envelope
against the hand-written layer's 25.6%, all of the latter domain types.
cycle.go is allocated the single largest slice (30.6%) and delivers ZERO: the
hard ceiling drops its whole section. runPayrollCycleAll, the hand-written
BuildPayslip and the real Upsert never reach the agent.
The second fixture is this repo, "how does explore allocate its output budget
across files", where scripts/agent-eval/*.mjs take 71.8% against tools.ts's
18.5% despite scoring 4.6x lower. It reads the live index, so its assertions
are relative rather than fixed percentages.
- scripts/agent-eval/probe-allocation.mjs — per-file budget-share probe,
driving the CG-4 diagnostic through a JSONL sidecar so it measures the
shipping allocator. Fixture entries are hermetic (copy + re-index per run,
verified byte-identical across runs); exits 1 while any assertion fails.
- scripts/agent-eval/allocation-fixtures.json — both fixtures declared, with
the 2026-08-03 baselines.
- __tests__/explore-allocation-1500.test.ts — fixture-shape assertions green
today; the allocation assertions held as `it.fails` so the suite stays green
while the bug is open and goes RED the moment it is fixed.
Also documented and deliberately left unfixed: runPayrollCycleAll's
`s.store.Upsert` edge resolves to the GENERATED Store.Upsert, not the
hand-written one — same-name method resolution across two packages picks the
wrong receiver. It is upstream of the allocation bug, so it belongs with
CG-10's scoring work.
Refs #1500
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Colby McHenryandClaude Opus 5
bd86ad2061