fix(kernel): CRLF docstring parity — JS multiline ^ anchors after \r, regex crate's (?m)^ is \n-only (#1329)

On CRLF checkouts (every Windows autocrlf clone) the JS reference's
block-continuation strip /^\s*\*\s?/gm finds a line start after the \r and
its greedy \s* consumes the \n, leaving a bare \r in the docstring; the
kernel's (?m)^ pass matched after \n only and kept \r\n. Caught by the O2
Windows VM leg (6 kernel-tsjs-parity failures), reproduced on macOS by
CRLF-converting the fixtures.

js_multiline_strip now replicates the JS anchor set (\n, \r, U+2028, U+2029)
for all five line-marker passes; CRLF variants of every torture fixture are
pinned in kernel-tsjs-parity, derived in-memory so nothing can normalize
them away.

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Colby Mchenry
2026-07-17 01:28:21 -05:00
committed by GitHub
co-authored by Claude Fable 5
parent 9e18ac2125
commit 5e329adc28
3 changed files with 93 additions and 10 deletions
+19
View File
@@ -125,6 +125,25 @@ describe.skipIf(!kernelBuilt)('kernel TS/JS extraction parity', () => {
assertParity(rel, fs.readFileSync(file, 'utf8'), 'typescript');
});
// Every torture fixture again with CRLF line endings — the shape every
// Windows autocrlf checkout has. Derived in memory (not a checked-in CRLF
// file) so no platform or editor can silently normalize it away. Pins the
// JS-multiline-^ semantics in the kernel's docstring cleaning: JS `^`/m
// anchors after \r too, so the block-continuation `\s*` eats the `\n` and
// the cleaned docstring keeps a bare `\r` (caught on the Windows VM leg of
// the O2 gate; diverged in the kernel until docstring.rs mirrored it).
it.each([
['torture.tsx', 'tsx'],
['torture.js', 'javascript'],
['Torture.java', 'java'],
['torture.py', 'python'],
['torture.go', 'go'],
] as const)('torture fixture CRLF parity: %s', (name, lang) => {
const file = path.join(FIXTURE_DIR, name);
const crlf = fs.readFileSync(file, 'utf8').replace(/(?<!\r)\n/g, '\r\n');
assertParity(`fixtures/${name} (crlf)`, crlf, lang);
});
it('files with parse errors defer to the wasm extractor (recovery is encoding-dependent)', () => {
// tree-sitter error RECOVERY differs between UTF-8 (native) and UTF-16
// (web-tree-sitter) parsing — same grammar, same core version — so the