perf(kernel): direct-to-store decode — buffers flow to the store worker, main thread never materializes nodes

Kernel-routed files ship their flat tables from the parse worker to the
store worker as buffers (tryKernelExtractRaw → kernelBuffers on the
result → KernelStoreBundle); the store worker decodes and finalizes
(finalizeStoreBundle shared with the object path so filter semantics
can never drift). Files with applicable framework extract() hooks keep
the decoded path; the no-writer fallback materializes via
materializeKernelResult. Byte-identical dumps re-verified on dubbo,
excalidraw, express, gson; full suite green (2,467).

Measurement (plan §4d): dubbo's parse-loop wall is 94% store-writer
busy time — the many-core fresh-index wall is single-writer SQLite
ingest, not extraction or main-thread work. d2s improves the writer
lane ~11% (structured-clone deserialization avoided on the writer) and
frees the main thread; the remaining many-core gap is a
store-architecture arc (deferred index builds, multi-file
transactions, buffer→bind), out of the kernel project's scope.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby McHenry
2026-07-16 23:38:32 -05:00
co-authored by Claude Fable 5
parent 03d54e47a1
commit 28068fa0f1
7 changed files with 277 additions and 39 deletions
+17
View File
@@ -276,6 +276,23 @@ export interface ExtractionResult {
/** Extraction duration in milliseconds */
durationMs: number;
/**
* Deferred-decode transport (native kernel, bulk-index path): when present,
* `nodes`/`edges`/`unresolvedReferences` are EMPTY and the file's tables
* ride as flat buffers to be decoded at the store boundary (the store
* worker), so the MAIN thread never materializes per-node objects.
* `kernelCounts` carries the table sizes for bookkeeping. Decode into a
* plain result with `materializeKernelResult` (src/extraction/kernel).
*/
kernelBuffers?: {
meta: Uint8Array;
nodes: Uint8Array;
edges: Uint8Array;
refs: Uint8Array;
arena: Uint8Array;
};
kernelCounts?: { nodes: number; edges: number; refs: number };
}
/**