docs(kernel): §7a.8 cFnPtr calibration — strip rewrite killed by measurement, fuse-then-link is step 1 (#1363)
Three measurements before any port. The stripCStyle split('') rewrite
(byte-identical segment-builder) measured 1.0× on 15.1M chars of linux C
— V8's ~73MB/s scan rate IS the cost, and 78s ≈ 4 strips/file × that
rate: the lever is the redundancy, not the scanner. Rewrite reverted;
the differential oracle test ships so any future rewrite stays pinned
byte-identical. E's regexes alone run at ~46MB/s (~30s of its 95s; the
rest is per-match logic and slicing).
Re-ordered attack recorded in §7a.8: step 1 = TS fuse-then-link refactor
(strip once per file, collect raw matches + declared-type tables,
text-free global linking; ≈ −70-90s, parity via collector insertion
order + the §7a.4 probe-hash gate); step 2 = native per-file extractor
behind the same boundary (raw disk text — no preParse interaction).
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
co-authored by
Claude Fable 5
Claude Fable 5
parent
971a5a0483
commit
b877db617c
@@ -0,0 +1,115 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { stripCommentsForRegex } from '../src/resolution/strip-comments';
|
||||
|
||||
/**
|
||||
* The pre-optimization split('')-based stripCStyle, kept verbatim as the
|
||||
* ORACLE: the rewritten segment-builder must be byte-identical on every
|
||||
* input (the C fn-pointer synthesizer's regexes run over this text, and any
|
||||
* divergence would silently change synthesized edges).
|
||||
*/
|
||||
function referenceStripCStyle(src: string, allowSingleQuoteStrings: boolean): string {
|
||||
const out = src.split('');
|
||||
let i = 0;
|
||||
const n = src.length;
|
||||
const blankRange = (buf: string[], start: number, end: number): void => {
|
||||
for (let k = start; k < end; k++) {
|
||||
buf[k] = src[k] === '\n' ? '\n' : ' ';
|
||||
}
|
||||
};
|
||||
while (i < n) {
|
||||
const c = src[i]!;
|
||||
const c2 = src[i + 1] ?? '';
|
||||
if (c === '/' && c2 === '*') {
|
||||
const start = i;
|
||||
i += 2;
|
||||
while (i < n && !(src[i] === '*' && src[i + 1] === '/')) i++;
|
||||
if (i < n) i += 2;
|
||||
blankRange(out, start, i);
|
||||
continue;
|
||||
}
|
||||
if (c === '/' && c2 === '/') {
|
||||
const start = i;
|
||||
while (i < n && src[i] !== '\n') i++;
|
||||
blankRange(out, start, i);
|
||||
continue;
|
||||
}
|
||||
if (c === '"' || (allowSingleQuoteStrings && c === "'") || c === '`') {
|
||||
const quote = c;
|
||||
i++;
|
||||
while (i < n && src[i] !== quote) {
|
||||
if (src[i] === '\\' && i + 1 < n) {
|
||||
i += 2;
|
||||
continue;
|
||||
}
|
||||
if (quote !== '`' && src[i] === '\n') break;
|
||||
i++;
|
||||
}
|
||||
if (i < n && src[i] === quote) i++;
|
||||
continue;
|
||||
}
|
||||
i++;
|
||||
}
|
||||
return out.join('');
|
||||
}
|
||||
|
||||
const FIXTURES: Array<[string, string]> = [
|
||||
['plain code, no comments', 'int main(void) {\n\treturn a / b;\n}\n'],
|
||||
['block comment', 'int x; /* a comment\nspanning lines */ int y;\n'],
|
||||
['line comment', 'int x; // trailing\nint y;\n'],
|
||||
['comment markers inside string', 'const char *s = "/* not a comment */ // nor this";\nint z;\n'],
|
||||
['string inside comment', '/* "a string" inside */ int q;\n'],
|
||||
['unterminated block comment', 'int a;\n/* runs to the end'],
|
||||
['unterminated string', 'const char *s = "no close\nint b; /* real comment */\n'],
|
||||
['escape at end of string', 'const char *s = "ends with backslash \\\\";\nint c;\n'],
|
||||
['escape as last char of file', 'const char *s = "\\'],
|
||||
['star at last char', 'int d; /*'],
|
||||
['slash at last char', 'int e; /'],
|
||||
['crlf line comment', 'int f; // comment\r\nint g;\r\n'],
|
||||
['unicode in comment', 'int h; /* café résumé — dash */\nint i;\n'],
|
||||
['astral chars in comment', 'int j; /* 🚀🎉 emoji */\nint k;\n'],
|
||||
['unicode in string', 'const char *s = "café 🚀";\nint l;\n'],
|
||||
['nested-looking block', '/* outer /* inner */ int m;\n'],
|
||||
['comment right after string', '"str"/*c*/int n;\n'],
|
||||
['backtick template (js mode relevance)', 'const t = `multi\nline ${x} // not comment`;\nint o;\n'],
|
||||
['single quotes with escapes', "char c = '\\''; // char literal\nint p;\n"],
|
||||
['empty input', ''],
|
||||
['only a newline', '\n'],
|
||||
['only a comment', '/*x*/'],
|
||||
];
|
||||
|
||||
describe('stripCStyle segment-builder vs split-based oracle', () => {
|
||||
for (const [name, src] of FIXTURES) {
|
||||
it(`fixture: ${name} (c mode)`, () => {
|
||||
expect(stripCommentsForRegex(src, 'c')).toBe(referenceStripCStyle(src, false));
|
||||
});
|
||||
it(`fixture: ${name} (js mode, single-quote strings on)`, () => {
|
||||
expect(stripCommentsForRegex(src, 'javascript')).toBe(referenceStripCStyle(src, true));
|
||||
});
|
||||
}
|
||||
|
||||
it('randomized differential (seeded, 500 cases)', () => {
|
||||
// Tiny deterministic LCG — no Math.random in tests that must reproduce.
|
||||
let seed = 0x2fn;
|
||||
const rand = (max: number): number => {
|
||||
seed = (seed * 6364136223846793005n + 1442695040888963407n) & 0xffffffffffffffffn;
|
||||
return Number(seed % BigInt(max));
|
||||
};
|
||||
const ATOMS = ['/*', '*/', '//', '\n', '"', "'", '`', '\\', 'x', ' ', '/', '*', 'é', '🚀', '\r\n', 'int a;'];
|
||||
for (let caseN = 0; caseN < 500; caseN++) {
|
||||
let s = '';
|
||||
const len = rand(40);
|
||||
for (let k = 0; k < len; k++) s += ATOMS[rand(ATOMS.length)]!;
|
||||
expect(stripCommentsForRegex(s, 'c'), `c-mode case ${caseN}: ${JSON.stringify(s)}`).toBe(
|
||||
referenceStripCStyle(s, false)
|
||||
);
|
||||
expect(stripCommentsForRegex(s, 'javascript'), `js-mode case ${caseN}: ${JSON.stringify(s)}`).toBe(
|
||||
referenceStripCStyle(s, true)
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
it('comment-free input returns the identical string (zero-copy path)', () => {
|
||||
const src = 'static int add(int a, int b) {\n\treturn a + b;\n}\n';
|
||||
expect(stripCommentsForRegex(src, 'c')).toBe(src);
|
||||
});
|
||||
});
|
||||
@@ -904,6 +904,44 @@ each ~16min), then the fix in two cadence iterations:
|
||||
without full-park folds — e.g. passive-checkpoint nudges at the recycle
|
||||
boundary) > backpressure byte volume > recreate.
|
||||
|
||||
#### 7a.8 cFnPtr calibration round (2026-07-18) — the 230s decomposed; fuse-then-link is step 1
|
||||
|
||||
Three quick measurements before any port, two of them killing assumptions:
|
||||
|
||||
- **JS strip rewrite: killed by measurement.** stripCStyle's `split('')`
|
||||
looked like allocator pathology; a segment-builder rewrite (byte-identical,
|
||||
pinned by `__tests__/strip-cstyle-differential.test.ts` — kept as the
|
||||
oracle for any future rewrite) measured **1.0×** on 15.1M chars of linux
|
||||
C. V8's scan rate is the honest cost: **~73MB/s**, and 283k strips ≈ 4
|
||||
strips/file × ~20KB × that rate ≈ the observed 78s. The strip lever is
|
||||
the **4× redundancy** (all-or-nothing cache declined at 6–7GB → every
|
||||
sweep re-strips), not the scanner.
|
||||
- **E-stage regexes alone: ~46MB/s → ~30s of E's 95s.** DISPATCH_RE +
|
||||
ARRAY_DISPATCH_RE over stripped kernel/ text yield 1,112 matches / 15.1M
|
||||
chars. The other ~65s is per-match logic, body slicing, lineAt, and
|
||||
getNodesInFile. A native regex scan alone caps at −30s.
|
||||
- **Calibrated attack for the ~230s, re-ordered:**
|
||||
1. **Fuse-then-link refactor (TS, step 1):** one per-file extraction pass
|
||||
computes strip ONCE and collects {function macros, object macros,
|
||||
defined sets, struct fields, raw registration matches, raw dispatch
|
||||
matches, per-function declared-receiver types}; a text-free global
|
||||
linking pass then builds registries and edges. Kills the 4× strip
|
||||
(−~58s) + repeated reads (−~8s) + part of E's slicing overhead.
|
||||
Parity discipline: collectors insert in the same file order the
|
||||
global passes iterate today (Map insertion order = current registry
|
||||
order), FANOUT_CAP and match-evaluation order preserved per function;
|
||||
gate = edge-set hash vs the live kernel DB (§7a.4 probe) + linux dump
|
||||
sha. The chain/receiver resolution must be pre-collected as per-file
|
||||
declared-type tables so linking never touches text.
|
||||
2. **Native per-file extractor (step 2):** the same boundary then accepts
|
||||
a Rust implementation of the per-file pass (raw text in, collected
|
||||
records out — no preParse interaction; the synthesizer reads raw disk
|
||||
text). Bug-for-bug regex semantics required; worth it only for the
|
||||
remaining ~100s of per-file scan+logic after step 1 lands.
|
||||
- Note for step 2 sizing: strip at native memchr rates (~500MB/s+) would
|
||||
be ~6-10s for the full corpus even before redundancy cuts — but marshal
|
||||
(UTF-16↔UTF-8 across napi) eats seconds at GB scale; batch the calls.
|
||||
|
||||
### 7b. Arc 3 — graph richness (forensics-backed; adopt cbm's real extras, skip inflation)
|
||||
Priority order, each gated by the standard A/B + node-explosion probes:
|
||||
1. **Test→subject edges** (first-class `tests` edges at index time; we compute covering
|
||||
|
||||
Reference in New Issue
Block a user