docs(playbook): first P7 agent A/B numbers (proshop, Sonnet, 2 runs/arm) and the plan's P7 status

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01REFyW9hmNrxhwN5wxRoAkC
2026-08-28 14:57:43 -05:00
parent bc45e9071d
commit 7f461269c4
2 changed files with 17 additions and 2 deletions
@@ -278,7 +278,7 @@ Status legend: ✅ done+validated · 🔬 hole identified · ⬜ not started.
 (Verify the exact supported set against `src/extraction/languages/` and
 `src/resolution/frameworks/` before starting — this table is a starting point.)
 | TypeScript/JS | Next.js (App Router + Pages Router) | page → `<Link>` / `router.push` / `redirect` → page; page load → data; client component → `'use server'` action → DB → `redirect`; `route.ts` handler → DB → response | R + S | ✅ 2026-08-28 (`frameworks/nextjs.ts`, `next-router-synthesizer.ts`, Steps `load` trigger + server-action crossing): fixture `__tests__/nextjs.test.ts` end to end (Screens `routed`, the push attributed back under its condition, the action's redirect); `leerob/next-saas-starter` indexes its pages and links. 🔬 agent A/B (`--model sonnet`, ≥2 runs/arm) not run yet |
-| TypeScript/JS | Express + React (MERN monorepo) | client `axios` / `fetch` literal path → own route → handler → Mongoose → response rows | S + R | ✅ 2026-08-28 (`tier-synthesizer.ts` `http-client` + Express mounts / chained `router.route()` / wrapped `const h = asyncHandler(…)` handlers / nested `package.json` detection): `bradtraversy/proshop_mern` 49 routes (19 pages + 30 endpoints), 23 client→route edges, every one spot-checked correct; `login → ⇢ POST /api/users/login → User.findOne → 401 rows → jwt.sign`. Node count stable across re-index. 🔬 A/B not run |
+| TypeScript/JS | Express + React (MERN monorepo) | client `axios` / `fetch` literal path → own route → handler → Mongoose → response rows | S + R | ✅ 2026-08-28 (`tier-synthesizer.ts` `http-client` + Express mounts / chained `router.route()` / wrapped `const h = asyncHandler(…)` handlers / nested `package.json` detection): `bradtraversy/proshop_mern` 49 routes (19 pages + 30 endpoints), 23 client→route edges, every one spot-checked correct; `login → ⇢ POST /api/users/login → User.findOne → 401 rows → jwt.sign`. Node count stable across re-index. **Agent A/B (2026-08-28, `run-all.sh`, Sonnet/high, 2 runs/arm, warm daemon, CLI shim on — 0 CLI leaks):** *"How does submitting the login form reach the database, and what does the API respond with when the password is wrong?"* — with: **14s / 14s, 2 tool calls each, 0 Read, 0 Grep, 1 and 2 explores** (explore sufficiency: answered after 1 call in run 1, after 2 in run 2), answer names `submitHandler → login → POST /api/users/login → authUser → User.findOne → matchPassword → 401` end to end; without: **18s / 37s, 14 and 8 tool calls, 7 and 1 Read** (+7 Bash `cat`s in run 2, a subagent in run 1). Tokens processed 153k/165k vs 237k/369k. Pass bar met on the small repo (0 Read/Grep, faster, within 1–2 explores against a budget of 1). 🔬 medium/large (Ghost) and ≥3 prompts still to run |
 | TypeScript/JS | NestJS queues / events / sockets | `queue.add('job')` → `@Process('job')`; `emit('x')` → `@OnEvent('x')`; `socket.emit` → `@SubscribeMessage` and `server.emit` → `socket.on` | S | ✅ 2026-08-28 (`tier-synthesizer.ts` `queue-job`, `event-bus`): `nestjs/nest` `sample/26-queues` (`transcode` → `handleTranscode`) and `sample/30-event-emitter` (`order.created` → its listener) exact, 0 wrong edges after the `e2e/` and generic-event guards; `immich-app/immich` 0 edges — a generated SDK client and a wrapped queue API carry no literal, so silence (correct). 🔬 A/B not run |
 | Python | FastAPI (prefixed routers) | request → `Depends` → handler → session → `HTTPException` rows | R | ✅ 2026-08-28 `python.ts` `postExtract` composes `APIRouter(prefix=)` + literal `include_router(prefix=)`: `fastapi/full-stack-fastapi-template` 23 routes named `GET /items/{id}`, `POST /login/access-token` (were `GET /`); a computed mount prefix is skipped. 🔬 A/B not run |
 | C# | ASP.NET Minimal API endpoint groups | request → group handler → `ISender.Send` → `TypedResults` rows | R | ✅ 2026-08-28 `csharp.ts` handler-first `MapPost(Handler[, path])` under the class + `$"/api/{groupName}"` head (`jasontaylordev/CleanArchitecture` shape); fixture end to end in `ui-steps-api-servers.test.ts` (`PUT /api/TodoItems/{id}` → `204 · 400` rows); `jasontaylordev/CleanArchitecture` 10 routes, all the app's endpoints (were 0). 🔬 A/B not run |