Files

T

Colby McHenryandClaude Opus 5 bd86ad2061 test(explore): #1500 regression fixtures for budget allocation (CG-6)

Two permanent fixtures pinning the failure mode from issue #1500 — explore
spending its byte envelope on files that merely name-collide with the query.
BOTH FAIL TODAY, by design: they document the bug and become the pass gate
for CG-10 (scoring) + CG-12 (proportional allocation).

__tests__/fixtures/payroll-go/ — a synthetic Go service mirroring the
reporter's shape: generated FKIT CRUD beside a hand-written payroll use-case,
entered from an HTTP route. Half the generated tree carries ORDINARY names
detectable only by their `// Code generated ... DO NOT EDIT.` header (the
#1500 case, and end-to-end cover for CG-5); `payrollpb/*.pb.go` covers the
path-detectable channel. BuildPayslip, Upsert and Store each exist twice,
generated and hand-written. cycle.go sits above the whole-file window so it
clips; the generated files sit below it so they ship whole.

Asking "how does payroll cycle create and calculate payslips?" — naming none
of the answering symbols — the generated CRUD delivers 57.4% of the envelope
against the hand-written layer's 25.6%, all of the latter domain types.
cycle.go is allocated the single largest slice (30.6%) and delivers ZERO: the
hard ceiling drops its whole section. runPayrollCycleAll, the hand-written
BuildPayslip and the real Upsert never reach the agent.

The second fixture is this repo, "how does explore allocate its output budget
across files", where scripts/agent-eval/*.mjs take 71.8% against tools.ts's
18.5% despite scoring 4.6x lower. It reads the live index, so its assertions
are relative rather than fixed percentages.

- scripts/agent-eval/probe-allocation.mjs — per-file budget-share probe,
  driving the CG-4 diagnostic through a JSONL sidecar so it measures the
  shipping allocator. Fixture entries are hermetic (copy + re-index per run,
  verified byte-identical across runs); exits 1 while any assertion fails.
- scripts/agent-eval/allocation-fixtures.json — both fixtures declared, with
  the 2026-08-03 baselines.
- __tests__/explore-allocation-1500.test.ts — fixture-shape assertions green
  today; the allocation assertions held as `it.fails` so the suite stays green
  while the bug is open and goes RED the moment it is fixed.

Also documented and deliberately left unfixed: runPayrollCycleAll's
`s.store.Upsert` edge resolves to the GENERATED Store.Upsert, not the
hand-written one — same-name method resolution across two packages picks the
wrong receiver. It is upstream of the allocation bug, so it belongs with
CG-10's scoring work.

Refs #1500

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

2026-08-03 23:30:17 -05:00

4.8 KiB

Raw Blame History

payroll-go — the #1500 regression fixture

A synthetic Go service reproducing the repo shape from issue #1500: generated CRUD sitting beside the hand-written use-case that does the real work.

This tree is a fixture, not a program. It never compiles or runs — it exists to be indexed. Keep it valid, idiomatic Go anyway: the extractor's output is the whole point.

The shape

cmd/payrolld/main.go                        wires the service
internal/transport/httpapi/                 HTTP entry point → use-case
internal/usecase/payroll/                   ← THE ANSWER. Hand-written workflow:
  cycle.go                                    runPayrollCycleAll (227 lines)
  payslip_builder.go                          BuildPayslip — the actual pay calculation
  prorate.go
internal/domain/payroll/payslip.go          hand-written domain types
internal/store/payslipstore/store.go        the real Upsert
internal/platform/clock/clock.go

internal/gen/fkit/payroll/                  ← THE NOISE. Generated CRUD, ORDINARY names:
  payslip.go        CreatePayslip, GetPayslip, UpdatePayslip, a second BuildPayslip
  payroll_cycle.go  CreatePayrollCycle, PayrollCycleCreateRequest, …
  store.go          a second Upsert
  calculate.go      CalculatePayrollCycleTotals, CalculatePayslipNet, …
  dto.go
internal/gen/fkit/employee/, timesheet/     more generated CRUD
internal/gen/payrollpb/*.pb.go              generated, detectable by PATH

The chain the fixture is built around is runPayrollCycleAll → BuildPayslip → Upsert, entered from POST /v1/payroll/cycles/{cycleID}/run.

The three properties that make it a regression fixture

Generated files that only a CONTENT header betrays. The internal/gen/fkit/** files have ordinary names (payslip.go, store.go) and carry // Code generated by fkit v3.11.0. DO NOT EDIT.. Path-only detection misses every one of them — that is the #1500 case, and why CG-5 added the content check. The payrollpb/*.pb.go files cover the path-detectable channel beside them.
Deliberate name collisions. BuildPayslip, Upsert and Store each exist twice, once generated and once hand-written. The generated layer also name-collides on every term of the question below — CreatePayslip, PayrollCycleCreateRequest, CalculatePayrollCycleTotals — so a scorer that rewards incidental name matches surfaces the CRUD path.
A size split that drives the render mode. cycle.go is deliberately over the whole-file window (227 lines) so it falls through to clipped clusters; the generated files are deliberately under it so they ship whole. Allocation follows file size, not relevance. __tests__/explore-allocation-1500.test.ts pins both sides of that split — if you edit these files, keep it.

The assertion

Query: "how does payroll cycle create and calculate payslips?" — an architecture question that names none of the symbols that answer it. The budget should concentrate on the hand-written workflow. As of 2026-08-03 it does not:

	allocated	delivered
hand-written	48.4%	25.6% (all of it domain types)
generated CRUD	39.9%	57.4%

cycle.go is allocated the single largest slice (7,052 chars, 30.6%) and delivers zero — the hard ceiling drops its whole section. payslip_builder.go (rank #8) never renders at all. runPayrollCycleAll, the hand-written BuildPayslip and the real Upsert never reach the agent.

Running it

npm run build
node scripts/agent-eval/probe-allocation.mjs payroll-go   # exits 1 today, by design
npx vitest run __tests__/explore-allocation-1500.test.ts  # green today, by design

The probe reports per-file budget share against scripts/agent-eval/allocation-fixtures.json. The vitest suite pins the fixture's shape and holds the allocation assertion as it.fails — green while the bug is open, red the moment it is fixed. See docs/design/explore-budget-allocation.md.

Known finding: the chain's `Upsert` edge resolves to the generated store

runPayrollCycleAll calls s.store.Upsert(ctx, slip), where s.store is a *payslipstore.Store. The graph resolves that edge to internal/gen/fkit/payroll/store.go — the generated Store.Upsert — not the hand-written one. Same-name method resolution across two packages that both define Store.Upsert picks the wrong receiver.

This is a resolution defect, not a budget one, and it is left unfixed on purpose: it is upstream of the allocation bug (a wrong edge pulls the generated store into the subgraph and inflates its score), so it belongs with the scoring work in CG-10 rather than here. The test asserts only that the workflow reaches an Upsert, so tightening the resolver later will not break the fixture.

4.8 KiB Raw Blame History