Remove .codegraph/config.json and the entire config surface. CodeGraph now indexes every file whose extension maps to a supported language and respects .gitignore everywhere — git repos via git itself, non-git projects via the `ignore` library (root + nested .gitignore files, the same way git does). - Remove CodeGraphConfig/DEFAULT_CONFIG, src/config.ts, and the public config API (the `config` option on init, getConfig/updateConfig/getConfigPath). - Derive the source-file allowlist from EXTENSION_MAP (isSourceFile); maxFileSize is now a constant. Drop the .codegraphignore marker. - Behavior change: committed, non-gitignored dirs (vendor/, a committed dist/) are now indexed — .gitignore is the single source of truth. Earlier inert fields (languages, frameworks, extractDocstrings, trackCallSites, customPatterns) and their dead helpers are removed as part of this. Resolves #283. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
5b71a89574
commit
f6772dac7c
@@ -29,6 +29,37 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|||||||
grammar, and `web/core`, `web/modules/contrib`, `web/themes/contrib` are
|
grammar, and `web/core`, `web/modules/contrib`, `web/themes/contrib` are
|
||||||
excluded by default. Resolves [#268](https://github.com/colbymchenry/codegraph/issues/268).
|
excluded by default. Resolves [#268](https://github.com/colbymchenry/codegraph/issues/268).
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **Zero-config indexing that respects `.gitignore`.** CodeGraph no longer has a
|
||||||
|
config file. It indexes every file whose extension maps to a supported language
|
||||||
|
and honors your `.gitignore` everywhere: in git repos via git itself, and in
|
||||||
|
non-git projects (e.g. a freshly-scaffolded app before `git init`) by reading
|
||||||
|
`.gitignore` files directly — root and nested, the same way git does (via the
|
||||||
|
`ignore` library, so negation/anchoring/nested rules all behave correctly). To
|
||||||
|
keep something out of the graph, add it to `.gitignore`. **Behavior change:**
|
||||||
|
committed files that are *not* gitignored are now indexed even under `vendor/`,
|
||||||
|
`Pods/`, or a committed `dist/` — previously a hardcoded exclude list skipped
|
||||||
|
those names; now `.gitignore` is the single source of truth. Resolves
|
||||||
|
[#283](https://github.com/colbymchenry/codegraph/issues/283).
|
||||||
|
|
||||||
|
### Removed
|
||||||
|
- **`.codegraph/config.json` and the entire config surface.** Every field was
|
||||||
|
either inert or now redundant with `.gitignore`:
|
||||||
|
- `languages`/`frameworks` never affected indexing (languages are detected per
|
||||||
|
file from extensions; frameworks are auto-detected). `languages` was also
|
||||||
|
broken — its validator only knew the original 8 languages, so setting it to
|
||||||
|
anything newer (C#, PHP, Ruby, C/C++, Swift, Kotlin, Dart, Vue, Scala, Lua, …)
|
||||||
|
threw `Invalid configuration format`.
|
||||||
|
- `extractDocstrings`/`trackCallSites`/`customPatterns` were never read by any
|
||||||
|
extractor.
|
||||||
|
- `include` is now derived from the supported language extensions, `exclude` is
|
||||||
|
replaced by `.gitignore`, and `maxFileSize` (1 MB) is a constant.
|
||||||
|
|
||||||
|
**Breaking (library API):** the `CodeGraphConfig` type, the `config` option on
|
||||||
|
`CodeGraph.init()`, and the `getConfig()`/`updateConfig()`/`getConfigPath`
|
||||||
|
exports are gone. Existing `.codegraph/config.json` files are simply ignored.
|
||||||
|
The `.codegraphignore` marker is no longer supported — use `.gitignore`.
|
||||||
|
|
||||||
## [0.9.1] - 2026-05-21
|
## [0.9.1] - 2026-05-21
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|||||||
@@ -418,28 +418,23 @@ cg.close();
|
|||||||
|
|
||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
The `.codegraph/config.json` file controls indexing:
|
There isn't any — CodeGraph is zero-config. It indexes every file whose
|
||||||
|
extension maps to a [supported language](#supported-languages) and **respects
|
||||||
|
your `.gitignore`**: in git repos via git itself, and in non-git projects by
|
||||||
|
reading `.gitignore` files directly (root and nested, the same way git would).
|
||||||
|
|
||||||
```json
|
What that means in practice:
|
||||||
{
|
|
||||||
"version": 1,
|
|
||||||
"languages": ["typescript", "javascript"],
|
|
||||||
"exclude": ["node_modules/**", "dist/**", "build/**", "*.min.js"],
|
|
||||||
"frameworks": [],
|
|
||||||
"maxFileSize": 1048576,
|
|
||||||
"extractDocstrings": true,
|
|
||||||
"trackCallSites": true
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
| Option | Description | Default |
|
- Anything git ignores — `node_modules`, build output, secrets in `.env` — is
|
||||||
|--------|-------------|---------|
|
never indexed. **To keep something out of the graph, add it to `.gitignore`.**
|
||||||
| `languages` | Languages to index (auto-detected if empty) | `[]` |
|
- There's no config file to write or keep in sync, and nothing to wire up per
|
||||||
| `exclude` | Glob patterns to ignore | `["node_modules/**", ...]` |
|
language: support is automatic from the file extension.
|
||||||
| `frameworks` | Framework hints for better resolution | `[]` |
|
- Files larger than 1 MB are skipped (generated bundles, minified JS, vendored
|
||||||
| `maxFileSize` | Skip files larger than this (bytes) | `1048576` (1MB) |
|
blobs) — they cost parse budget for no useful symbols.
|
||||||
| `extractDocstrings` | Extract docstrings from code | `true` |
|
|
||||||
| `trackCallSites` | Track call site locations | `true` |
|
> Committed files that aren't gitignored *are* indexed, even under `vendor/` or a
|
||||||
|
> committed `dist/`. If you commit a dependency or build directory you don't want
|
||||||
|
> in the graph, add it to `.gitignore`.
|
||||||
|
|
||||||
## Supported Languages
|
## Supported Languages
|
||||||
|
|
||||||
|
|||||||
@@ -9,10 +9,9 @@ import * as fs from 'fs';
|
|||||||
import * as path from 'path';
|
import * as path from 'path';
|
||||||
import * as os from 'os';
|
import * as os from 'os';
|
||||||
import { CodeGraph } from '../src';
|
import { CodeGraph } from '../src';
|
||||||
import { extractFromSource, scanDirectory, shouldIncludeFile } from '../src/extraction';
|
import { extractFromSource, scanDirectory } from '../src/extraction';
|
||||||
import { detectLanguage, isLanguageSupported, getSupportedLanguages, initGrammars, loadAllGrammars } from '../src/extraction/grammars';
|
import { detectLanguage, isLanguageSupported, getSupportedLanguages, initGrammars, loadAllGrammars } from '../src/extraction/grammars';
|
||||||
import { normalizePath } from '../src/utils';
|
import { normalizePath } from '../src/utils';
|
||||||
import { DEFAULT_CONFIG } from '../src/types';
|
|
||||||
|
|
||||||
beforeAll(async () => {
|
beforeAll(async () => {
|
||||||
await initGrammars();
|
await initGrammars();
|
||||||
@@ -3003,39 +3002,57 @@ describe('Directory Exclusion', () => {
|
|||||||
cleanupTempDir(tempDir);
|
cleanupTempDir(tempDir);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should exclude node_modules directories', () => {
|
it('should exclude directories listed in .gitignore', () => {
|
||||||
// Create structure: src/index.ts + node_modules/pkg/index.js
|
// Create structure: src/index.ts + node_modules/pkg/index.js, gitignore node_modules
|
||||||
const srcDir = path.join(tempDir, 'src');
|
const srcDir = path.join(tempDir, 'src');
|
||||||
const nmDir = path.join(tempDir, 'node_modules', 'pkg');
|
const nmDir = path.join(tempDir, 'node_modules', 'pkg');
|
||||||
fs.mkdirSync(srcDir, { recursive: true });
|
fs.mkdirSync(srcDir, { recursive: true });
|
||||||
fs.mkdirSync(nmDir, { recursive: true });
|
fs.mkdirSync(nmDir, { recursive: true });
|
||||||
fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;');
|
fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;');
|
||||||
fs.writeFileSync(path.join(nmDir, 'index.js'), 'module.exports = {};');
|
fs.writeFileSync(path.join(nmDir, 'index.js'), 'module.exports = {};');
|
||||||
|
fs.writeFileSync(path.join(tempDir, '.gitignore'), 'node_modules/\n');
|
||||||
|
|
||||||
const config = { ...DEFAULT_CONFIG, rootDir: tempDir };
|
const files = scanDirectory(tempDir);
|
||||||
const files = scanDirectory(tempDir, config);
|
|
||||||
|
|
||||||
expect(files).toContain('src/index.ts');
|
expect(files).toContain('src/index.ts');
|
||||||
expect(files.every((f) => !f.includes('node_modules'))).toBe(true);
|
expect(files.every((f) => !f.includes('node_modules'))).toBe(true);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should exclude nested node_modules directories', () => {
|
it('should exclude nested node_modules via a root .gitignore', () => {
|
||||||
// Create structure: packages/app/node_modules/pkg/index.js
|
// A trailing-slash pattern with no leading slash matches at any depth.
|
||||||
const srcDir = path.join(tempDir, 'packages', 'app', 'src');
|
const srcDir = path.join(tempDir, 'packages', 'app', 'src');
|
||||||
const nmDir = path.join(tempDir, 'packages', 'app', 'node_modules', 'pkg');
|
const nmDir = path.join(tempDir, 'packages', 'app', 'node_modules', 'pkg');
|
||||||
fs.mkdirSync(srcDir, { recursive: true });
|
fs.mkdirSync(srcDir, { recursive: true });
|
||||||
fs.mkdirSync(nmDir, { recursive: true });
|
fs.mkdirSync(nmDir, { recursive: true });
|
||||||
fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;');
|
fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;');
|
||||||
fs.writeFileSync(path.join(nmDir, 'index.js'), 'module.exports = {};');
|
fs.writeFileSync(path.join(nmDir, 'index.js'), 'module.exports = {};');
|
||||||
|
fs.writeFileSync(path.join(tempDir, '.gitignore'), 'node_modules/\n');
|
||||||
|
|
||||||
const config = { ...DEFAULT_CONFIG, rootDir: tempDir };
|
const files = scanDirectory(tempDir);
|
||||||
const files = scanDirectory(tempDir, config);
|
|
||||||
|
|
||||||
expect(files).toContain('packages/app/src/index.ts');
|
expect(files).toContain('packages/app/src/index.ts');
|
||||||
expect(files.every((f) => !f.includes('node_modules'))).toBe(true);
|
expect(files.every((f) => !f.includes('node_modules'))).toBe(true);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should exclude .git directories', () => {
|
it('should apply a nested .gitignore only to its own subtree', () => {
|
||||||
|
const appSrc = path.join(tempDir, 'app', 'src');
|
||||||
|
fs.mkdirSync(appSrc, { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(appSrc, 'keep.ts'), 'export const a = 1;');
|
||||||
|
fs.writeFileSync(path.join(appSrc, 'skip.ts'), 'export const b = 2;');
|
||||||
|
fs.writeFileSync(path.join(tempDir, 'app', '.gitignore'), 'src/skip.ts\n');
|
||||||
|
// A sibling with the same name outside app/ must NOT be ignored.
|
||||||
|
const otherDir = path.join(tempDir, 'other', 'src');
|
||||||
|
fs.mkdirSync(otherDir, { recursive: true });
|
||||||
|
fs.writeFileSync(path.join(otherDir, 'skip.ts'), 'export const c = 3;');
|
||||||
|
|
||||||
|
const files = scanDirectory(tempDir);
|
||||||
|
|
||||||
|
expect(files).toContain('app/src/keep.ts');
|
||||||
|
expect(files).not.toContain('app/src/skip.ts');
|
||||||
|
expect(files).toContain('other/src/skip.ts');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should always skip .git directories', () => {
|
||||||
const srcDir = path.join(tempDir, 'src');
|
const srcDir = path.join(tempDir, 'src');
|
||||||
const gitDir = path.join(tempDir, '.git', 'objects');
|
const gitDir = path.join(tempDir, '.git', 'objects');
|
||||||
fs.mkdirSync(srcDir, { recursive: true });
|
fs.mkdirSync(srcDir, { recursive: true });
|
||||||
@@ -3043,8 +3060,7 @@ describe('Directory Exclusion', () => {
|
|||||||
fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;');
|
fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;');
|
||||||
fs.writeFileSync(path.join(gitDir, 'pack.ts'), 'export const y = 2;');
|
fs.writeFileSync(path.join(gitDir, 'pack.ts'), 'export const y = 2;');
|
||||||
|
|
||||||
const config = { ...DEFAULT_CONFIG, rootDir: tempDir };
|
const files = scanDirectory(tempDir);
|
||||||
const files = scanDirectory(tempDir, config);
|
|
||||||
|
|
||||||
expect(files).toContain('src/index.ts');
|
expect(files).toContain('src/index.ts');
|
||||||
expect(files.every((f) => !f.includes('.git'))).toBe(true);
|
expect(files.every((f) => !f.includes('.git'))).toBe(true);
|
||||||
@@ -3055,29 +3071,12 @@ describe('Directory Exclusion', () => {
|
|||||||
fs.mkdirSync(srcDir, { recursive: true });
|
fs.mkdirSync(srcDir, { recursive: true });
|
||||||
fs.writeFileSync(path.join(srcDir, 'Button.tsx'), 'export function Button() {}');
|
fs.writeFileSync(path.join(srcDir, 'Button.tsx'), 'export function Button() {}');
|
||||||
|
|
||||||
const config = { ...DEFAULT_CONFIG, rootDir: tempDir };
|
const files = scanDirectory(tempDir);
|
||||||
const files = scanDirectory(tempDir, config);
|
|
||||||
|
|
||||||
expect(files.length).toBe(1);
|
expect(files.length).toBe(1);
|
||||||
expect(files[0]).toBe('src/components/Button.tsx');
|
expect(files[0]).toBe('src/components/Button.tsx');
|
||||||
expect(files[0]).not.toContain('\\');
|
expect(files[0]).not.toContain('\\');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should respect .codegraphignore marker', () => {
|
|
||||||
const srcDir = path.join(tempDir, 'src');
|
|
||||||
const vendorDir = path.join(tempDir, 'vendor');
|
|
||||||
fs.mkdirSync(srcDir, { recursive: true });
|
|
||||||
fs.mkdirSync(vendorDir, { recursive: true });
|
|
||||||
fs.writeFileSync(path.join(srcDir, 'index.ts'), 'export const x = 1;');
|
|
||||||
fs.writeFileSync(path.join(vendorDir, 'lib.ts'), 'export const y = 2;');
|
|
||||||
fs.writeFileSync(path.join(vendorDir, '.codegraphignore'), '');
|
|
||||||
|
|
||||||
const config = { ...DEFAULT_CONFIG, rootDir: tempDir };
|
|
||||||
const files = scanDirectory(tempDir, config);
|
|
||||||
|
|
||||||
expect(files).toContain('src/index.ts');
|
|
||||||
expect(files.every((f) => !f.includes('vendor'))).toBe(true);
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('Git Submodules', () => {
|
describe('Git Submodules', () => {
|
||||||
@@ -3124,8 +3123,7 @@ describe('Git Submodules', () => {
|
|||||||
);
|
);
|
||||||
git(mainDir, 'commit', '-q', '-m', 'add submodule');
|
git(mainDir, 'commit', '-q', '-m', 'add submodule');
|
||||||
|
|
||||||
const config = { ...DEFAULT_CONFIG, rootDir: mainDir };
|
const files = scanDirectory(mainDir);
|
||||||
const files = scanDirectory(mainDir, config);
|
|
||||||
|
|
||||||
expect(files).toContain('app.ts');
|
expect(files).toContain('app.ts');
|
||||||
expect(files).toContain('libs/lib/lib.ts');
|
expect(files).toContain('libs/lib/lib.ts');
|
||||||
@@ -3173,8 +3171,7 @@ describe('Nested non-submodule git repos', () => {
|
|||||||
git(path.join(root, 'sub_repo2'), 'init', '-q');
|
git(path.join(root, 'sub_repo2'), 'init', '-q');
|
||||||
fs.writeFileSync(path.join(sub2, 'two.ts'), 'export const two = 2;');
|
fs.writeFileSync(path.join(sub2, 'two.ts'), 'export const two = 2;');
|
||||||
|
|
||||||
const config = { ...DEFAULT_CONFIG, rootDir: root };
|
const files = scanDirectory(root);
|
||||||
const files = scanDirectory(root, config);
|
|
||||||
|
|
||||||
// Both committed and untracked source from the nested repos must be found.
|
// Both committed and untracked source from the nested repos must be found.
|
||||||
expect(files).toContain('sub_repo1/src/one.ts');
|
expect(files).toContain('sub_repo1/src/one.ts');
|
||||||
@@ -3197,8 +3194,7 @@ describe('Nested non-submodule git repos', () => {
|
|||||||
fs.writeFileSync(path.join(sub, 'real.ts'), 'export const real = 1;');
|
fs.writeFileSync(path.join(sub, 'real.ts'), 'export const real = 1;');
|
||||||
fs.writeFileSync(path.join(sub, 'generated.ts'), 'export const generated = 1;');
|
fs.writeFileSync(path.join(sub, 'generated.ts'), 'export const generated = 1;');
|
||||||
|
|
||||||
const config = { ...DEFAULT_CONFIG, rootDir: root };
|
const files = scanDirectory(root);
|
||||||
const files = scanDirectory(root, config);
|
|
||||||
|
|
||||||
expect(files).toContain('sub_repo/src/real.ts');
|
expect(files).toContain('sub_repo/src/real.ts');
|
||||||
expect(files).not.toContain('sub_repo/src/generated.ts');
|
expect(files).not.toContain('sub_repo/src/generated.ts');
|
||||||
|
|||||||
@@ -9,8 +9,7 @@ import * as fs from 'fs';
|
|||||||
import * as path from 'path';
|
import * as path from 'path';
|
||||||
import * as os from 'os';
|
import * as os from 'os';
|
||||||
import { CodeGraph } from '../src';
|
import { CodeGraph } from '../src';
|
||||||
import { DEFAULT_CONFIG, Node, Edge } from '../src/types';
|
import { Node, Edge } from '../src/types';
|
||||||
import { loadConfig, saveConfig } from '../src/config';
|
|
||||||
import { isInitialized, getCodeGraphDir, validateDirectory } from '../src/directory';
|
import { isInitialized, getCodeGraphDir, validateDirectory } from '../src/directory';
|
||||||
import { DatabaseConnection, getDatabasePath } from '../src/db';
|
import { DatabaseConnection, getDatabasePath } from '../src/db';
|
||||||
|
|
||||||
@@ -60,41 +59,12 @@ describe('CodeGraph Foundation', () => {
|
|||||||
cg.close();
|
cg.close();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should create config.json with defaults', () => {
|
|
||||||
const cg = CodeGraph.initSync(tempDir);
|
|
||||||
|
|
||||||
const configPath = path.join(getCodeGraphDir(tempDir), 'config.json');
|
|
||||||
expect(fs.existsSync(configPath)).toBe(true);
|
|
||||||
|
|
||||||
const config = cg.getConfig();
|
|
||||||
expect(config.version).toBe(DEFAULT_CONFIG.version);
|
|
||||||
expect(config.include).toEqual(DEFAULT_CONFIG.include);
|
|
||||||
expect(config.exclude).toEqual(DEFAULT_CONFIG.exclude);
|
|
||||||
|
|
||||||
cg.close();
|
|
||||||
});
|
|
||||||
|
|
||||||
it('should throw if already initialized', () => {
|
it('should throw if already initialized', () => {
|
||||||
const cg = CodeGraph.initSync(tempDir);
|
const cg = CodeGraph.initSync(tempDir);
|
||||||
cg.close();
|
cg.close();
|
||||||
|
|
||||||
expect(() => CodeGraph.initSync(tempDir)).toThrow(/already initialized/i);
|
expect(() => CodeGraph.initSync(tempDir)).toThrow(/already initialized/i);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should accept custom config options', () => {
|
|
||||||
const cg = CodeGraph.initSync(tempDir, {
|
|
||||||
config: {
|
|
||||||
maxFileSize: 500000,
|
|
||||||
extractDocstrings: false,
|
|
||||||
},
|
|
||||||
});
|
|
||||||
|
|
||||||
const config = cg.getConfig();
|
|
||||||
expect(config.maxFileSize).toBe(500000);
|
|
||||||
expect(config.extractDocstrings).toBe(false);
|
|
||||||
|
|
||||||
cg.close();
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('Opening Projects', () => {
|
describe('Opening Projects', () => {
|
||||||
@@ -112,17 +82,6 @@ describe('CodeGraph Foundation', () => {
|
|||||||
it('should throw if not initialized', () => {
|
it('should throw if not initialized', () => {
|
||||||
expect(() => CodeGraph.openSync(tempDir)).toThrow(/not initialized/i);
|
expect(() => CodeGraph.openSync(tempDir)).toThrow(/not initialized/i);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should preserve configuration across open/close', () => {
|
|
||||||
const cg1 = CodeGraph.initSync(tempDir, {
|
|
||||||
config: { maxFileSize: 123456 },
|
|
||||||
});
|
|
||||||
cg1.close();
|
|
||||||
|
|
||||||
const cg2 = CodeGraph.openSync(tempDir);
|
|
||||||
expect(cg2.getConfig().maxFileSize).toBe(123456);
|
|
||||||
cg2.close();
|
|
||||||
});
|
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('Static Methods', () => {
|
describe('Static Methods', () => {
|
||||||
@@ -182,31 +141,6 @@ describe('CodeGraph Foundation', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('Configuration', () => {
|
|
||||||
it('should load and merge config with defaults', () => {
|
|
||||||
const cg = CodeGraph.initSync(tempDir);
|
|
||||||
cg.close();
|
|
||||||
|
|
||||||
const config = loadConfig(tempDir);
|
|
||||||
expect(config.version).toBe(DEFAULT_CONFIG.version);
|
|
||||||
expect(config.rootDir).toBe(path.resolve(tempDir));
|
|
||||||
});
|
|
||||||
|
|
||||||
it('should update configuration', () => {
|
|
||||||
const cg = CodeGraph.initSync(tempDir);
|
|
||||||
|
|
||||||
cg.updateConfig({ maxFileSize: 999999 });
|
|
||||||
|
|
||||||
expect(cg.getConfig().maxFileSize).toBe(999999);
|
|
||||||
|
|
||||||
cg.close();
|
|
||||||
|
|
||||||
// Verify persistence
|
|
||||||
const config = loadConfig(tempDir);
|
|
||||||
expect(config.maxFileSize).toBe(999999);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
|
|
||||||
describe('Directory Management', () => {
|
describe('Directory Management', () => {
|
||||||
it('should validate directory structure', () => {
|
it('should validate directory structure', () => {
|
||||||
const cg = CodeGraph.initSync(tempDir);
|
const cg = CodeGraph.initSync(tempDir);
|
||||||
|
|||||||
+18
-72
@@ -15,9 +15,7 @@ import * as os from 'os';
|
|||||||
import { FileLock } from '../src/utils';
|
import { FileLock } from '../src/utils';
|
||||||
import CodeGraph from '../src/index';
|
import CodeGraph from '../src/index';
|
||||||
import { ToolHandler, tools } from '../src/mcp/tools';
|
import { ToolHandler, tools } from '../src/mcp/tools';
|
||||||
import { shouldIncludeFile, scanDirectory } from '../src/extraction';
|
import { scanDirectory, isSourceFile } from '../src/extraction';
|
||||||
import { shouldIncludeFile as configShouldInclude } from '../src/config';
|
|
||||||
import { CodeGraphConfig, DEFAULT_CONFIG } from '../src/types';
|
|
||||||
import { DatabaseConnection, getDatabasePath } from '../src/db';
|
import { DatabaseConnection, getDatabasePath } from '../src/db';
|
||||||
import { QueryBuilder } from '../src/db/queries';
|
import { QueryBuilder } from '../src/db/queries';
|
||||||
|
|
||||||
@@ -298,58 +296,24 @@ describe('Atomic Writes', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
describe('Glob Matching (picomatch)', () => {
|
describe('Source file detection (isSourceFile)', () => {
|
||||||
const makeConfig = (include: string[], exclude: string[]): CodeGraphConfig => ({
|
it('selects files by supported extension', () => {
|
||||||
...DEFAULT_CONFIG,
|
expect(isSourceFile('src/index.ts')).toBe(true);
|
||||||
rootDir: '/test',
|
expect(isSourceFile('src/deep/nested/file.ts')).toBe(true);
|
||||||
include,
|
expect(isSourceFile('src/component.tsx')).toBe(true);
|
||||||
exclude,
|
expect(isSourceFile('lib/util.js')).toBe(true);
|
||||||
|
expect(isSourceFile('src/main.py')).toBe(true);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should match standard glob patterns in extraction', () => {
|
it('rejects unsupported extensions and extensionless files', () => {
|
||||||
const config = makeConfig(['**/*.ts'], ['node_modules/**']);
|
expect(isSourceFile('src/component.css')).toBe(false);
|
||||||
|
expect(isSourceFile('README.md')).toBe(false);
|
||||||
expect(shouldIncludeFile('src/index.ts', config)).toBe(true);
|
expect(isSourceFile('Makefile')).toBe(false);
|
||||||
expect(shouldIncludeFile('src/deep/nested/file.ts', config)).toBe(true);
|
expect(isSourceFile('.gitignore')).toBe(false);
|
||||||
expect(shouldIncludeFile('src/index.js', config)).toBe(false);
|
|
||||||
expect(shouldIncludeFile('node_modules/lib/index.ts', config)).toBe(false);
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should match standard glob patterns in config', () => {
|
it('matches regardless of leading dot directories', () => {
|
||||||
const config = makeConfig(['**/*.py'], ['__pycache__/**']);
|
expect(isSourceFile('.hidden/index.ts')).toBe(true);
|
||||||
|
|
||||||
expect(configShouldInclude('src/main.py', config)).toBe(true);
|
|
||||||
expect(configShouldInclude('src/main.ts', config)).toBe(false);
|
|
||||||
expect(configShouldInclude('__pycache__/module.py', config)).toBe(false);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('should handle complex glob patterns correctly', () => {
|
|
||||||
const config = makeConfig(['src/**/*.{ts,tsx}', 'lib/**/*.js'], []);
|
|
||||||
|
|
||||||
expect(shouldIncludeFile('src/component.ts', config)).toBe(true);
|
|
||||||
expect(shouldIncludeFile('src/component.tsx', config)).toBe(true);
|
|
||||||
expect(shouldIncludeFile('lib/util.js', config)).toBe(true);
|
|
||||||
expect(shouldIncludeFile('src/component.css', config)).toBe(false);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('should handle patterns that previously caused ReDoS', () => {
|
|
||||||
// This pattern would cause catastrophic backtracking with hand-rolled regex
|
|
||||||
const evilPattern = '**/**/**/**/**/**/**/**/**/**/**/**/**/**/a';
|
|
||||||
const config = makeConfig([evilPattern], []);
|
|
||||||
|
|
||||||
const start = Date.now();
|
|
||||||
// This should return quickly, not hang
|
|
||||||
shouldIncludeFile('x/x/x/x/x/x/x/x/x/x/x/x/x/x/b', config);
|
|
||||||
const elapsed = Date.now() - start;
|
|
||||||
|
|
||||||
// Should complete in under 100ms, not seconds
|
|
||||||
expect(elapsed).toBeLessThan(100);
|
|
||||||
});
|
|
||||||
|
|
||||||
it('should handle dot files correctly', () => {
|
|
||||||
const config = makeConfig(['**/*.ts'], []);
|
|
||||||
|
|
||||||
expect(shouldIncludeFile('.hidden/index.ts', config)).toBe(true);
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -464,15 +428,9 @@ describe('Symlink Cycle Detection', () => {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
const config: CodeGraphConfig = {
|
|
||||||
...DEFAULT_CONFIG,
|
|
||||||
rootDir: tempDir,
|
|
||||||
include: ['**/*.ts'],
|
|
||||||
exclude: [],
|
|
||||||
};
|
|
||||||
|
|
||||||
// This should complete without hanging
|
// This should complete without hanging
|
||||||
const files = scanDirectory(tempDir, config);
|
const files = scanDirectory(tempDir);
|
||||||
|
|
||||||
// Should find the real file but not loop infinitely
|
// Should find the real file but not loop infinitely
|
||||||
expect(files).toContain('src/index.ts');
|
expect(files).toContain('src/index.ts');
|
||||||
@@ -496,14 +454,8 @@ describe('Symlink Cycle Detection', () => {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
const config: CodeGraphConfig = {
|
|
||||||
...DEFAULT_CONFIG,
|
|
||||||
rootDir: tempDir,
|
|
||||||
include: ['**/*.ts'],
|
|
||||||
exclude: [],
|
|
||||||
};
|
|
||||||
|
|
||||||
const files = scanDirectory(tempDir, config);
|
const files = scanDirectory(tempDir);
|
||||||
|
|
||||||
// Should find files from both the real dir and via the symlink
|
// Should find files from both the real dir and via the symlink
|
||||||
// But deduplicate since they resolve to the same real path
|
// But deduplicate since they resolve to the same real path
|
||||||
@@ -521,15 +473,9 @@ describe('Symlink Cycle Detection', () => {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
const config: CodeGraphConfig = {
|
|
||||||
...DEFAULT_CONFIG,
|
|
||||||
rootDir: tempDir,
|
|
||||||
include: ['**/*.ts'],
|
|
||||||
exclude: [],
|
|
||||||
};
|
|
||||||
|
|
||||||
// Should not throw
|
// Should not throw
|
||||||
const files = scanDirectory(tempDir, config);
|
const files = scanDirectory(tempDir);
|
||||||
expect(files).toContain('src/valid.ts');
|
expect(files).toContain('src/valid.ts');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -281,11 +281,11 @@ describe('Sync Module', () => {
|
|||||||
expect(nodes.length).toBe(0);
|
expect(nodes.length).toBe(0);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should skip files not matching config', async () => {
|
it('should skip files with unsupported extensions', async () => {
|
||||||
// Create a .js file which doesn't match **/*.ts
|
// A .txt file has no supported grammar, so sync must not index it.
|
||||||
fs.writeFileSync(
|
fs.writeFileSync(
|
||||||
path.join(testDir, 'src', 'ignored.js'),
|
path.join(testDir, 'src', 'notes.txt'),
|
||||||
`function ignored() {}`
|
`just some notes`
|
||||||
);
|
);
|
||||||
|
|
||||||
const result = await cg.sync();
|
const result = await cg.sync();
|
||||||
|
|||||||
@@ -12,7 +12,6 @@ import * as path from 'path';
|
|||||||
import * as os from 'os';
|
import * as os from 'os';
|
||||||
import { watchDisabledReason } from '../src/sync/watch-policy';
|
import { watchDisabledReason } from '../src/sync/watch-policy';
|
||||||
import { FileWatcher } from '../src/sync/watcher';
|
import { FileWatcher } from '../src/sync/watcher';
|
||||||
import type { CodeGraphConfig } from '../src/types';
|
|
||||||
|
|
||||||
describe('watchDisabledReason', () => {
|
describe('watchDisabledReason', () => {
|
||||||
it('returns a reason when CODEGRAPH_NO_WATCH=1', () => {
|
it('returns a reason when CODEGRAPH_NO_WATCH=1', () => {
|
||||||
@@ -63,18 +62,6 @@ describe('watchDisabledReason', () => {
|
|||||||
describe('FileWatcher honors the watch policy', () => {
|
describe('FileWatcher honors the watch policy', () => {
|
||||||
let testDir: string;
|
let testDir: string;
|
||||||
|
|
||||||
const baseConfig: CodeGraphConfig = {
|
|
||||||
version: 1,
|
|
||||||
rootDir: '.',
|
|
||||||
include: ['**/*.ts'],
|
|
||||||
exclude: ['**/node_modules/**'],
|
|
||||||
languages: [],
|
|
||||||
frameworks: [],
|
|
||||||
maxFileSize: 1024 * 1024,
|
|
||||||
extractDocstrings: true,
|
|
||||||
trackCallSites: true,
|
|
||||||
};
|
|
||||||
|
|
||||||
afterEach(() => {
|
afterEach(() => {
|
||||||
delete process.env.CODEGRAPH_NO_WATCH;
|
delete process.env.CODEGRAPH_NO_WATCH;
|
||||||
if (testDir && fs.existsSync(testDir)) {
|
if (testDir && fs.existsSync(testDir)) {
|
||||||
@@ -87,7 +74,7 @@ describe('FileWatcher honors the watch policy', () => {
|
|||||||
process.env.CODEGRAPH_NO_WATCH = '1';
|
process.env.CODEGRAPH_NO_WATCH = '1';
|
||||||
|
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn);
|
const watcher = new FileWatcher(testDir, syncFn);
|
||||||
|
|
||||||
expect(watcher.start()).toBe(false);
|
expect(watcher.start()).toBe(false);
|
||||||
expect(watcher.isActive()).toBe(false);
|
expect(watcher.isActive()).toBe(false);
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ import * as fs from 'fs';
|
|||||||
import * as path from 'path';
|
import * as path from 'path';
|
||||||
import * as os from 'os';
|
import * as os from 'os';
|
||||||
import { FileWatcher } from '../src/sync/watcher';
|
import { FileWatcher } from '../src/sync/watcher';
|
||||||
import type { CodeGraphConfig } from '../src/types';
|
|
||||||
import CodeGraph from '../src/index';
|
import CodeGraph from '../src/index';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -34,18 +33,6 @@ function waitFor(
|
|||||||
describe('FileWatcher', () => {
|
describe('FileWatcher', () => {
|
||||||
let testDir: string;
|
let testDir: string;
|
||||||
|
|
||||||
const baseConfig: CodeGraphConfig = {
|
|
||||||
version: 1,
|
|
||||||
rootDir: '.',
|
|
||||||
include: ['**/*.ts', '**/*.js'],
|
|
||||||
exclude: ['**/node_modules/**', '**/dist/**'],
|
|
||||||
languages: [],
|
|
||||||
frameworks: [],
|
|
||||||
maxFileSize: 1024 * 1024,
|
|
||||||
extractDocstrings: true,
|
|
||||||
trackCallSites: true,
|
|
||||||
};
|
|
||||||
|
|
||||||
beforeEach(() => {
|
beforeEach(() => {
|
||||||
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-watcher-'));
|
testDir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-watcher-'));
|
||||||
// Create a source file so the directory isn't empty
|
// Create a source file so the directory isn't empty
|
||||||
@@ -63,7 +50,7 @@ describe('FileWatcher', () => {
|
|||||||
describe('start/stop lifecycle', () => {
|
describe('start/stop lifecycle', () => {
|
||||||
it('should start and stop without errors', () => {
|
it('should start and stop without errors', () => {
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn);
|
const watcher = new FileWatcher(testDir, syncFn);
|
||||||
|
|
||||||
const started = watcher.start();
|
const started = watcher.start();
|
||||||
expect(started).toBe(true);
|
expect(started).toBe(true);
|
||||||
@@ -75,7 +62,7 @@ describe('FileWatcher', () => {
|
|||||||
|
|
||||||
it('should be idempotent on double start', () => {
|
it('should be idempotent on double start', () => {
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn);
|
const watcher = new FileWatcher(testDir, syncFn);
|
||||||
|
|
||||||
expect(watcher.start()).toBe(true);
|
expect(watcher.start()).toBe(true);
|
||||||
expect(watcher.start()).toBe(true); // Should not throw
|
expect(watcher.start()).toBe(true); // Should not throw
|
||||||
@@ -86,7 +73,7 @@ describe('FileWatcher', () => {
|
|||||||
|
|
||||||
it('should be idempotent on double stop', () => {
|
it('should be idempotent on double stop', () => {
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn);
|
const watcher = new FileWatcher(testDir, syncFn);
|
||||||
|
|
||||||
watcher.start();
|
watcher.start();
|
||||||
watcher.stop();
|
watcher.stop();
|
||||||
@@ -98,7 +85,7 @@ describe('FileWatcher', () => {
|
|||||||
describe('debounced sync', () => {
|
describe('debounced sync', () => {
|
||||||
it('should trigger sync after file change', async () => {
|
it('should trigger sync after file change', async () => {
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 });
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 });
|
const watcher = new FileWatcher(testDir, syncFn, { debounceMs: 200 });
|
||||||
|
|
||||||
watcher.start();
|
watcher.start();
|
||||||
|
|
||||||
@@ -114,7 +101,7 @@ describe('FileWatcher', () => {
|
|||||||
|
|
||||||
it('should debounce rapid changes into a single sync', async () => {
|
it('should debounce rapid changes into a single sync', async () => {
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 1, durationMs: 10 });
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 500 });
|
const watcher = new FileWatcher(testDir, syncFn, { debounceMs: 500 });
|
||||||
|
|
||||||
watcher.start();
|
watcher.start();
|
||||||
|
|
||||||
@@ -140,7 +127,7 @@ describe('FileWatcher', () => {
|
|||||||
describe('filtering', () => {
|
describe('filtering', () => {
|
||||||
it('should ignore files not matching include patterns', async () => {
|
it('should ignore files not matching include patterns', async () => {
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 });
|
const watcher = new FileWatcher(testDir, syncFn, { debounceMs: 200 });
|
||||||
|
|
||||||
watcher.start();
|
watcher.start();
|
||||||
|
|
||||||
@@ -160,7 +147,7 @@ describe('FileWatcher', () => {
|
|||||||
|
|
||||||
it('should ignore .codegraph directory changes', async () => {
|
it('should ignore .codegraph directory changes', async () => {
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 0, durationMs: 0 });
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn, { debounceMs: 200 });
|
const watcher = new FileWatcher(testDir, syncFn, { debounceMs: 200 });
|
||||||
|
|
||||||
watcher.start();
|
watcher.start();
|
||||||
|
|
||||||
@@ -185,7 +172,7 @@ describe('FileWatcher', () => {
|
|||||||
it('should call onSyncComplete after successful sync', async () => {
|
it('should call onSyncComplete after successful sync', async () => {
|
||||||
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 2, durationMs: 50 });
|
const syncFn = vi.fn().mockResolvedValue({ filesChanged: 2, durationMs: 50 });
|
||||||
const onSyncComplete = vi.fn();
|
const onSyncComplete = vi.fn();
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn, {
|
const watcher = new FileWatcher(testDir, syncFn, {
|
||||||
debounceMs: 200,
|
debounceMs: 200,
|
||||||
onSyncComplete,
|
onSyncComplete,
|
||||||
});
|
});
|
||||||
@@ -203,7 +190,7 @@ describe('FileWatcher', () => {
|
|||||||
it('should call onSyncError when sync throws', async () => {
|
it('should call onSyncError when sync throws', async () => {
|
||||||
const syncFn = vi.fn().mockRejectedValue(new Error('sync failed'));
|
const syncFn = vi.fn().mockRejectedValue(new Error('sync failed'));
|
||||||
const onSyncError = vi.fn();
|
const onSyncError = vi.fn();
|
||||||
const watcher = new FileWatcher(testDir, baseConfig, syncFn, {
|
const watcher = new FileWatcher(testDir, syncFn, {
|
||||||
debounceMs: 200,
|
debounceMs: 200,
|
||||||
onSyncError,
|
onSyncError,
|
||||||
});
|
});
|
||||||
|
|||||||
Generated
+10
@@ -13,6 +13,7 @@
|
|||||||
"commander": "^14.0.2",
|
"commander": "^14.0.2",
|
||||||
"fast-string-width": "^3.0.2",
|
"fast-string-width": "^3.0.2",
|
||||||
"fast-wrap-ansi": "^0.2.0",
|
"fast-wrap-ansi": "^0.2.0",
|
||||||
|
"ignore": "^7.0.5",
|
||||||
"jsonc-parser": "^3.3.1",
|
"jsonc-parser": "^3.3.1",
|
||||||
"picomatch": "^4.0.3",
|
"picomatch": "^4.0.3",
|
||||||
"sisteransi": "^1.0.5",
|
"sisteransi": "^1.0.5",
|
||||||
@@ -1145,6 +1146,15 @@
|
|||||||
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/ignore": {
|
||||||
|
"version": "7.0.5",
|
||||||
|
"resolved": "https://registry.npmjs.org/ignore/-/ignore-7.0.5.tgz",
|
||||||
|
"integrity": "sha512-Hs59xBNfUIunMFgWAbGX5cq6893IbWg4KnrjbYwX3tx0ztorVgTDA6B2sxf8ejHJ4wz8BqGUMYlnzNBer5NvGg==",
|
||||||
|
"license": "MIT",
|
||||||
|
"engines": {
|
||||||
|
"node": ">= 4"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/jsonc-parser": {
|
"node_modules/jsonc-parser": {
|
||||||
"version": "3.3.1",
|
"version": "3.3.1",
|
||||||
"resolved": "https://registry.npmjs.org/jsonc-parser/-/jsonc-parser-3.3.1.tgz",
|
"resolved": "https://registry.npmjs.org/jsonc-parser/-/jsonc-parser-3.3.1.tgz",
|
||||||
|
|||||||
@@ -36,6 +36,7 @@
|
|||||||
"commander": "^14.0.2",
|
"commander": "^14.0.2",
|
||||||
"fast-string-width": "^3.0.2",
|
"fast-string-width": "^3.0.2",
|
||||||
"fast-wrap-ansi": "^0.2.0",
|
"fast-wrap-ansi": "^0.2.0",
|
||||||
|
"ignore": "^7.0.5",
|
||||||
"jsonc-parser": "^3.3.1",
|
"jsonc-parser": "^3.3.1",
|
||||||
"picomatch": "^4.0.3",
|
"picomatch": "^4.0.3",
|
||||||
"sisteransi": "^1.0.5",
|
"sisteransi": "^1.0.5",
|
||||||
|
|||||||
-297
@@ -1,297 +0,0 @@
|
|||||||
/**
|
|
||||||
* Configuration Management
|
|
||||||
*
|
|
||||||
* Load, save, and validate CodeGraph configuration.
|
|
||||||
*/
|
|
||||||
|
|
||||||
import * as fs from 'fs';
|
|
||||||
import * as path from 'path';
|
|
||||||
import picomatch from 'picomatch';
|
|
||||||
import { CodeGraphConfig, DEFAULT_CONFIG, Language, NodeKind } from './types';
|
|
||||||
import { normalizePath } from './utils';
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Configuration filename
|
|
||||||
*/
|
|
||||||
export const CONFIG_FILENAME = 'config.json';
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get the config file path for a project
|
|
||||||
*/
|
|
||||||
export function getConfigPath(projectRoot: string): string {
|
|
||||||
return path.join(projectRoot, '.codegraph', CONFIG_FILENAME);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Check if a regex pattern is safe from ReDoS attacks.
|
|
||||||
*
|
|
||||||
* Rejects patterns with nested quantifiers (e.g., (a+)+, (a*)*) which
|
|
||||||
* are the primary source of catastrophic backtracking. Also rejects
|
|
||||||
* excessively long patterns and validates compilability.
|
|
||||||
*/
|
|
||||||
function isSafeRegex(pattern: string): boolean {
|
|
||||||
// Reject excessively long patterns
|
|
||||||
if (pattern.length > 500) return false;
|
|
||||||
|
|
||||||
// Reject nested quantifiers: (...)+ followed by +, *, or {
|
|
||||||
// These are the primary cause of catastrophic backtracking
|
|
||||||
if (/([+*}])\s*[+*{]/.test(pattern)) return false;
|
|
||||||
if (/\([^)]*[+*][^)]*\)[+*{]/.test(pattern)) return false;
|
|
||||||
|
|
||||||
// Verify the pattern is a valid regex
|
|
||||||
try {
|
|
||||||
new RegExp(pattern);
|
|
||||||
return true;
|
|
||||||
} catch {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Validate a configuration object
|
|
||||||
*/
|
|
||||||
export function validateConfig(config: unknown): config is CodeGraphConfig {
|
|
||||||
if (typeof config !== 'object' || config === null) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
const c = config as Record<string, unknown>;
|
|
||||||
|
|
||||||
// Required fields
|
|
||||||
if (typeof c.version !== 'number') return false;
|
|
||||||
if (typeof c.rootDir !== 'string') return false;
|
|
||||||
if (!Array.isArray(c.include)) return false;
|
|
||||||
if (!Array.isArray(c.exclude)) return false;
|
|
||||||
if (!Array.isArray(c.languages)) return false;
|
|
||||||
if (!Array.isArray(c.frameworks)) return false;
|
|
||||||
if (typeof c.maxFileSize !== 'number') return false;
|
|
||||||
if (typeof c.extractDocstrings !== 'boolean') return false;
|
|
||||||
if (typeof c.trackCallSites !== 'boolean') return false;
|
|
||||||
|
|
||||||
// Validate include/exclude are string arrays
|
|
||||||
if (!c.include.every((p) => typeof p === 'string')) return false;
|
|
||||||
if (!c.exclude.every((p) => typeof p === 'string')) return false;
|
|
||||||
|
|
||||||
// Validate languages
|
|
||||||
const validLanguages: Language[] = [
|
|
||||||
'typescript',
|
|
||||||
'javascript',
|
|
||||||
'python',
|
|
||||||
'go',
|
|
||||||
'rust',
|
|
||||||
'java',
|
|
||||||
'svelte',
|
|
||||||
'unknown',
|
|
||||||
];
|
|
||||||
if (!c.languages.every((l) => validLanguages.includes(l as Language))) return false;
|
|
||||||
|
|
||||||
// Validate frameworks
|
|
||||||
for (const fw of c.frameworks) {
|
|
||||||
if (typeof fw !== 'object' || fw === null) return false;
|
|
||||||
const framework = fw as Record<string, unknown>;
|
|
||||||
if (typeof framework.name !== 'string') return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Validate custom patterns if present
|
|
||||||
if (c.customPatterns !== undefined) {
|
|
||||||
if (!Array.isArray(c.customPatterns)) return false;
|
|
||||||
for (const pattern of c.customPatterns) {
|
|
||||||
if (typeof pattern !== 'object' || pattern === null) return false;
|
|
||||||
const p = pattern as Record<string, unknown>;
|
|
||||||
if (typeof p.name !== 'string') return false;
|
|
||||||
if (typeof p.pattern !== 'string') return false;
|
|
||||||
if (typeof p.kind !== 'string') return false;
|
|
||||||
|
|
||||||
// Validate regex is compilable and reject patterns with known ReDoS risks
|
|
||||||
if (!isSafeRegex(p.pattern)) return false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Merge configuration with defaults
|
|
||||||
*/
|
|
||||||
function mergeConfig(
|
|
||||||
defaults: CodeGraphConfig,
|
|
||||||
overrides: Partial<CodeGraphConfig>
|
|
||||||
): CodeGraphConfig {
|
|
||||||
return {
|
|
||||||
version: overrides.version ?? defaults.version,
|
|
||||||
rootDir: overrides.rootDir ?? defaults.rootDir,
|
|
||||||
include: overrides.include ?? defaults.include,
|
|
||||||
exclude: overrides.exclude ?? defaults.exclude,
|
|
||||||
languages: overrides.languages ?? defaults.languages,
|
|
||||||
frameworks: overrides.frameworks ?? defaults.frameworks,
|
|
||||||
maxFileSize: overrides.maxFileSize ?? defaults.maxFileSize,
|
|
||||||
extractDocstrings: overrides.extractDocstrings ?? defaults.extractDocstrings,
|
|
||||||
trackCallSites: overrides.trackCallSites ?? defaults.trackCallSites,
|
|
||||||
customPatterns: overrides.customPatterns ?? defaults.customPatterns,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Load configuration from a project
|
|
||||||
*/
|
|
||||||
export function loadConfig(projectRoot: string): CodeGraphConfig {
|
|
||||||
const configPath = getConfigPath(projectRoot);
|
|
||||||
|
|
||||||
if (!fs.existsSync(configPath)) {
|
|
||||||
// Return default config with adjusted rootDir
|
|
||||||
return {
|
|
||||||
...DEFAULT_CONFIG,
|
|
||||||
rootDir: projectRoot,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const content = fs.readFileSync(configPath, 'utf-8');
|
|
||||||
const parsed = JSON.parse(content) as unknown;
|
|
||||||
|
|
||||||
// Merge with defaults to ensure all fields are present
|
|
||||||
const merged = mergeConfig(DEFAULT_CONFIG, parsed as Partial<CodeGraphConfig>);
|
|
||||||
merged.rootDir = projectRoot; // Always use actual project root
|
|
||||||
|
|
||||||
if (!validateConfig(merged)) {
|
|
||||||
throw new Error('Invalid configuration format');
|
|
||||||
}
|
|
||||||
|
|
||||||
return merged;
|
|
||||||
} catch (error) {
|
|
||||||
if (error instanceof SyntaxError) {
|
|
||||||
throw new Error(`Invalid JSON in config file: ${configPath}`);
|
|
||||||
}
|
|
||||||
throw error;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Save configuration to a project
|
|
||||||
*/
|
|
||||||
export function saveConfig(projectRoot: string, config: CodeGraphConfig): void {
|
|
||||||
const configPath = getConfigPath(projectRoot);
|
|
||||||
const dir = path.dirname(configPath);
|
|
||||||
|
|
||||||
// Ensure directory exists
|
|
||||||
if (!fs.existsSync(dir)) {
|
|
||||||
fs.mkdirSync(dir, { recursive: true });
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create a copy without rootDir (it's always derived from project path)
|
|
||||||
const toSave = { ...config };
|
|
||||||
delete (toSave as Partial<CodeGraphConfig>).rootDir;
|
|
||||||
|
|
||||||
const content = JSON.stringify(toSave, null, 2);
|
|
||||||
|
|
||||||
// Atomic write: write to temp file then rename to prevent partial/corrupt configs
|
|
||||||
const tmpPath = configPath + '.tmp';
|
|
||||||
fs.writeFileSync(tmpPath, content, 'utf-8');
|
|
||||||
fs.renameSync(tmpPath, configPath);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Create default configuration for a new project
|
|
||||||
*/
|
|
||||||
export function createDefaultConfig(projectRoot: string): CodeGraphConfig {
|
|
||||||
return {
|
|
||||||
...DEFAULT_CONFIG,
|
|
||||||
rootDir: projectRoot,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Update specific configuration values
|
|
||||||
*/
|
|
||||||
export function updateConfig(
|
|
||||||
projectRoot: string,
|
|
||||||
updates: Partial<CodeGraphConfig>
|
|
||||||
): CodeGraphConfig {
|
|
||||||
const current = loadConfig(projectRoot);
|
|
||||||
const updated = mergeConfig(current, updates);
|
|
||||||
updated.rootDir = projectRoot;
|
|
||||||
saveConfig(projectRoot, updated);
|
|
||||||
return updated;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Add patterns to include list
|
|
||||||
*/
|
|
||||||
export function addIncludePatterns(projectRoot: string, patterns: string[]): CodeGraphConfig {
|
|
||||||
const config = loadConfig(projectRoot);
|
|
||||||
const newPatterns = patterns.filter((p) => !config.include.includes(p));
|
|
||||||
config.include = [...config.include, ...newPatterns];
|
|
||||||
saveConfig(projectRoot, config);
|
|
||||||
return config;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Add patterns to exclude list
|
|
||||||
*/
|
|
||||||
export function addExcludePatterns(projectRoot: string, patterns: string[]): CodeGraphConfig {
|
|
||||||
const config = loadConfig(projectRoot);
|
|
||||||
const newPatterns = patterns.filter((p) => !config.exclude.includes(p));
|
|
||||||
config.exclude = [...config.exclude, ...newPatterns];
|
|
||||||
saveConfig(projectRoot, config);
|
|
||||||
return config;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Add a custom pattern
|
|
||||||
*/
|
|
||||||
export function addCustomPattern(
|
|
||||||
projectRoot: string,
|
|
||||||
name: string,
|
|
||||||
pattern: string,
|
|
||||||
kind: NodeKind
|
|
||||||
): CodeGraphConfig {
|
|
||||||
const config = loadConfig(projectRoot);
|
|
||||||
|
|
||||||
if (!config.customPatterns) {
|
|
||||||
config.customPatterns = [];
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check for duplicate name
|
|
||||||
const existing = config.customPatterns.find((p) => p.name === name);
|
|
||||||
if (existing) {
|
|
||||||
existing.pattern = pattern;
|
|
||||||
existing.kind = kind;
|
|
||||||
} else {
|
|
||||||
config.customPatterns.push({ name, pattern, kind });
|
|
||||||
}
|
|
||||||
|
|
||||||
saveConfig(projectRoot, config);
|
|
||||||
return config;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Check if a file path matches the include/exclude patterns
|
|
||||||
*/
|
|
||||||
export function shouldIncludeFile(filePath: string, config: CodeGraphConfig): boolean {
|
|
||||||
// Normalize to forward slashes so Windows backslash paths match glob patterns
|
|
||||||
filePath = normalizePath(filePath);
|
|
||||||
|
|
||||||
// Simple glob matching (for now, just check if any pattern matches)
|
|
||||||
// A full implementation would use a proper glob library
|
|
||||||
|
|
||||||
const matchesPattern = (pattern: string, filePath: string): boolean => {
|
|
||||||
return picomatch.isMatch(filePath, pattern, { dot: true });
|
|
||||||
};
|
|
||||||
|
|
||||||
// Check exclude patterns first
|
|
||||||
for (const pattern of config.exclude) {
|
|
||||||
if (matchesPattern(pattern, filePath)) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check include patterns
|
|
||||||
for (const pattern of config.include) {
|
|
||||||
if (matchesPattern(pattern, filePath)) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Default to not including if no pattern matches
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
@@ -94,6 +94,17 @@ export const EXTENSION_MAP: Record<string, Language> = {
|
|||||||
'.luau': 'luau',
|
'.luau': 'luau',
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether a file is one CodeGraph can parse, based purely on its extension.
|
||||||
|
* This is the single source of truth for "should we index this file" — derived
|
||||||
|
* from EXTENSION_MAP so parser support and indexing selection never drift.
|
||||||
|
*/
|
||||||
|
export function isSourceFile(filePath: string): boolean {
|
||||||
|
const dot = filePath.lastIndexOf('.');
|
||||||
|
if (dot < 0) return false;
|
||||||
|
return filePath.slice(dot).toLowerCase() in EXTENSION_MAP;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Caches for loaded grammars and parsers
|
* Caches for loaded grammars and parsers
|
||||||
*/
|
*/
|
||||||
|
|||||||
+79
-102
@@ -14,14 +14,13 @@ import {
|
|||||||
FileRecord,
|
FileRecord,
|
||||||
ExtractionResult,
|
ExtractionResult,
|
||||||
ExtractionError,
|
ExtractionError,
|
||||||
CodeGraphConfig,
|
|
||||||
} from '../types';
|
} from '../types';
|
||||||
import { QueryBuilder } from '../db/queries';
|
import { QueryBuilder } from '../db/queries';
|
||||||
import { extractFromSource } from './tree-sitter';
|
import { extractFromSource } from './tree-sitter';
|
||||||
import { detectLanguage, isLanguageSupported, initGrammars, loadGrammarsForLanguages } from './grammars';
|
import { detectLanguage, isSourceFile, isLanguageSupported, initGrammars, loadGrammarsForLanguages } from './grammars';
|
||||||
import { logDebug, logWarn } from '../errors';
|
import { logDebug, logWarn } from '../errors';
|
||||||
import { validatePathWithinRoot, normalizePath } from '../utils';
|
import { validatePathWithinRoot, normalizePath } from '../utils';
|
||||||
import picomatch from 'picomatch';
|
import ignore, { Ignore } from 'ignore';
|
||||||
import { detectFrameworks } from '../resolution/frameworks';
|
import { detectFrameworks } from '../resolution/frameworks';
|
||||||
import type { ResolutionContext } from '../resolution/types';
|
import type { ResolutionContext } from '../resolution/types';
|
||||||
|
|
||||||
@@ -94,36 +93,11 @@ export function hashContent(content: string): string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Check if a path matches any glob pattern (simplified)
|
* Skip files larger than this (bytes). Generated bundles, minified JS, and
|
||||||
|
* vendored blobs blow the WASM heap and the worker-recycle budget for no useful
|
||||||
|
* symbols. 1 MB covers essentially all hand-written source.
|
||||||
*/
|
*/
|
||||||
function matchesGlob(filePath: string, pattern: string): boolean {
|
const MAX_FILE_SIZE = 1024 * 1024;
|
||||||
filePath = normalizePath(filePath);
|
|
||||||
return picomatch.isMatch(filePath, pattern, { dot: true });
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Check if a file should be included based on config
|
|
||||||
*/
|
|
||||||
export function shouldIncludeFile(
|
|
||||||
filePath: string,
|
|
||||||
config: CodeGraphConfig
|
|
||||||
): boolean {
|
|
||||||
// Check exclude patterns first
|
|
||||||
for (const pattern of config.exclude) {
|
|
||||||
if (matchesGlob(filePath, pattern)) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check include patterns
|
|
||||||
for (const pattern of config.include) {
|
|
||||||
if (matchesGlob(filePath, pattern)) {
|
|
||||||
return true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Collect git-visible files (tracked + untracked, .gitignore-respected) from the
|
* Collect git-visible files (tracked + untracked, .gitignore-respected) from the
|
||||||
@@ -230,7 +204,7 @@ interface GitChanges {
|
|||||||
* Use `git status` to detect changed files instead of scanning every file.
|
* Use `git status` to detect changed files instead of scanning every file.
|
||||||
* Returns null on failure so callers fall back to full scan.
|
* Returns null on failure so callers fall back to full scan.
|
||||||
*/
|
*/
|
||||||
function getGitChangedFiles(rootDir: string, config: CodeGraphConfig): GitChanges | null {
|
function getGitChangedFiles(rootDir: string): GitChanges | null {
|
||||||
try {
|
try {
|
||||||
const output = execFileSync(
|
const output = execFileSync(
|
||||||
'git',
|
'git',
|
||||||
@@ -248,8 +222,8 @@ function getGitChangedFiles(rootDir: string, config: CodeGraphConfig): GitChange
|
|||||||
const statusCode = line.substring(0, 2);
|
const statusCode = line.substring(0, 2);
|
||||||
const filePath = normalizePath(line.substring(3));
|
const filePath = normalizePath(line.substring(3));
|
||||||
|
|
||||||
// Skip files that don't match include/exclude config
|
// Skip non-source files (git status already omits .gitignored paths).
|
||||||
if (!shouldIncludeFile(filePath, config)) continue;
|
if (!isSourceFile(filePath)) continue;
|
||||||
|
|
||||||
if (statusCode === '??') {
|
if (statusCode === '??') {
|
||||||
added.push(filePath);
|
added.push(filePath);
|
||||||
@@ -268,20 +242,14 @@ function getGitChangedFiles(rootDir: string, config: CodeGraphConfig): GitChange
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Marker file name that indicates a directory (and all children) should be skipped
|
* Recursively scan a directory for source files.
|
||||||
*/
|
|
||||||
const CODEGRAPH_IGNORE_MARKER = '.codegraphignore';
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Recursively scan directory for source files.
|
|
||||||
*
|
*
|
||||||
* In git repos, uses `git ls-files` to get the file list (inherently
|
* In git repos, uses `git ls-files` (inherently respects .gitignore at all
|
||||||
* respects .gitignore at all levels), then filters by config include patterns.
|
* levels), then keeps files with a supported source extension. For non-git
|
||||||
* Falls back to filesystem walk for non-git projects.
|
* projects, falls back to a filesystem walk that parses .gitignore itself.
|
||||||
*/
|
*/
|
||||||
export function scanDirectory(
|
export function scanDirectory(
|
||||||
rootDir: string,
|
rootDir: string,
|
||||||
config: CodeGraphConfig,
|
|
||||||
onProgress?: (current: number, file: string) => void
|
onProgress?: (current: number, file: string) => void
|
||||||
): string[] {
|
): string[] {
|
||||||
// Fast path: use git to get all visible files (respects .gitignore everywhere)
|
// Fast path: use git to get all visible files (respects .gitignore everywhere)
|
||||||
@@ -290,7 +258,7 @@ export function scanDirectory(
|
|||||||
const files: string[] = [];
|
const files: string[] = [];
|
||||||
let count = 0;
|
let count = 0;
|
||||||
for (const filePath of gitFiles) {
|
for (const filePath of gitFiles) {
|
||||||
if (shouldIncludeFile(filePath, config)) {
|
if (isSourceFile(filePath)) {
|
||||||
files.push(filePath);
|
files.push(filePath);
|
||||||
count++;
|
count++;
|
||||||
onProgress?.(count, filePath);
|
onProgress?.(count, filePath);
|
||||||
@@ -300,7 +268,7 @@ export function scanDirectory(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Fallback: walk filesystem for non-git projects
|
// Fallback: walk filesystem for non-git projects
|
||||||
return scanDirectoryWalk(rootDir, config, onProgress);
|
return scanDirectoryWalk(rootDir, onProgress);
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -309,7 +277,6 @@ export function scanDirectory(
|
|||||||
*/
|
*/
|
||||||
export async function scanDirectoryAsync(
|
export async function scanDirectoryAsync(
|
||||||
rootDir: string,
|
rootDir: string,
|
||||||
config: CodeGraphConfig,
|
|
||||||
onProgress?: (current: number, file: string) => void
|
onProgress?: (current: number, file: string) => void
|
||||||
): Promise<string[]> {
|
): Promise<string[]> {
|
||||||
const gitFiles = getGitVisibleFiles(rootDir);
|
const gitFiles = getGitVisibleFiles(rootDir);
|
||||||
@@ -317,7 +284,7 @@ export async function scanDirectoryAsync(
|
|||||||
const files: string[] = [];
|
const files: string[] = [];
|
||||||
let count = 0;
|
let count = 0;
|
||||||
for (const filePath of gitFiles) {
|
for (const filePath of gitFiles) {
|
||||||
if (shouldIncludeFile(filePath, config)) {
|
if (isSourceFile(filePath)) {
|
||||||
files.push(filePath);
|
files.push(filePath);
|
||||||
count++;
|
count++;
|
||||||
onProgress?.(count, filePath);
|
onProgress?.(count, filePath);
|
||||||
@@ -330,7 +297,7 @@ export async function scanDirectoryAsync(
|
|||||||
return files;
|
return files;
|
||||||
}
|
}
|
||||||
|
|
||||||
return scanDirectoryWalk(rootDir, config, onProgress);
|
return scanDirectoryWalk(rootDir, onProgress);
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -338,14 +305,44 @@ export async function scanDirectoryAsync(
|
|||||||
*/
|
*/
|
||||||
function scanDirectoryWalk(
|
function scanDirectoryWalk(
|
||||||
rootDir: string,
|
rootDir: string,
|
||||||
config: CodeGraphConfig,
|
|
||||||
onProgress?: (current: number, file: string) => void
|
onProgress?: (current: number, file: string) => void
|
||||||
): string[] {
|
): string[] {
|
||||||
const files: string[] = [];
|
const files: string[] = [];
|
||||||
let count = 0;
|
let count = 0;
|
||||||
const visitedDirs = new Set<string>();
|
const visitedDirs = new Set<string>();
|
||||||
|
|
||||||
function walk(dir: string): void {
|
// A .gitignore matcher scoped to the directory that declared it. Patterns in
|
||||||
|
// a nested .gitignore are relative to that directory, so we keep the dir
|
||||||
|
// alongside the matcher and test paths relative to it — mirroring how git
|
||||||
|
// applies .gitignore files at every level.
|
||||||
|
interface ScopedIgnore {
|
||||||
|
dir: string;
|
||||||
|
ig: Ignore;
|
||||||
|
}
|
||||||
|
|
||||||
|
const loadIgnore = (dir: string): ScopedIgnore | null => {
|
||||||
|
try {
|
||||||
|
const giPath = path.join(dir, '.gitignore');
|
||||||
|
if (fs.existsSync(giPath)) {
|
||||||
|
return { dir, ig: ignore().add(fs.readFileSync(giPath, 'utf-8')) };
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
// Unreadable .gitignore — treat as absent.
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
};
|
||||||
|
|
||||||
|
const isIgnored = (fullPath: string, isDir: boolean, matchers: ScopedIgnore[]): boolean => {
|
||||||
|
for (const { dir, ig } of matchers) {
|
||||||
|
let rel = normalizePath(path.relative(dir, fullPath));
|
||||||
|
if (!rel || rel.startsWith('..')) continue; // not under this matcher's dir
|
||||||
|
if (isDir) rel += '/'; // dir-only rules (e.g. `build/`) only match with the slash
|
||||||
|
if (ig.ignores(rel)) return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
};
|
||||||
|
|
||||||
|
function walk(dir: string, matchers: ScopedIgnore[]): void {
|
||||||
let realDir: string;
|
let realDir: string;
|
||||||
try {
|
try {
|
||||||
realDir = fs.realpathSync(dir);
|
realDir = fs.realpathSync(dir);
|
||||||
@@ -360,12 +357,9 @@ function scanDirectoryWalk(
|
|||||||
}
|
}
|
||||||
visitedDirs.add(realDir);
|
visitedDirs.add(realDir);
|
||||||
|
|
||||||
// Check for .codegraphignore marker file
|
// This directory's own .gitignore (if present) applies to everything below it.
|
||||||
const ignoreMarker = path.join(dir, CODEGRAPH_IGNORE_MARKER);
|
const own = loadIgnore(dir);
|
||||||
if (fs.existsSync(ignoreMarker)) {
|
const active = own ? [...matchers, own] : matchers;
|
||||||
logDebug('Skipping directory due to .codegraphignore marker', { dir });
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
let entries: fs.Dirent[];
|
let entries: fs.Dirent[];
|
||||||
try {
|
try {
|
||||||
@@ -376,6 +370,9 @@ function scanDirectoryWalk(
|
|||||||
}
|
}
|
||||||
|
|
||||||
for (const entry of entries) {
|
for (const entry of entries) {
|
||||||
|
// Never descend into git internals or our own data directory.
|
||||||
|
if (entry.name === '.git' || entry.name === '.codegraph') continue;
|
||||||
|
|
||||||
const fullPath = path.join(dir, entry.name);
|
const fullPath = path.join(dir, entry.name);
|
||||||
const relativePath = normalizePath(path.relative(rootDir, fullPath));
|
const relativePath = normalizePath(path.relative(rootDir, fullPath));
|
||||||
|
|
||||||
@@ -384,19 +381,11 @@ function scanDirectoryWalk(
|
|||||||
const realTarget = fs.realpathSync(fullPath);
|
const realTarget = fs.realpathSync(fullPath);
|
||||||
const stat = fs.statSync(realTarget);
|
const stat = fs.statSync(realTarget);
|
||||||
if (stat.isDirectory()) {
|
if (stat.isDirectory()) {
|
||||||
const dirPattern = relativePath + '/';
|
if (!isIgnored(fullPath, true, active)) {
|
||||||
let excluded = false;
|
walk(fullPath, active);
|
||||||
for (const pattern of config.exclude) {
|
|
||||||
if (matchesGlob(dirPattern, pattern) || matchesGlob(relativePath, pattern)) {
|
|
||||||
excluded = true;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (!excluded) {
|
|
||||||
walk(fullPath);
|
|
||||||
}
|
}
|
||||||
} else if (stat.isFile()) {
|
} else if (stat.isFile()) {
|
||||||
if (shouldIncludeFile(relativePath, config)) {
|
if (!isIgnored(fullPath, false, active) && isSourceFile(relativePath)) {
|
||||||
files.push(relativePath);
|
files.push(relativePath);
|
||||||
count++;
|
count++;
|
||||||
onProgress?.(count, relativePath);
|
onProgress?.(count, relativePath);
|
||||||
@@ -409,19 +398,11 @@ function scanDirectoryWalk(
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (entry.isDirectory()) {
|
if (entry.isDirectory()) {
|
||||||
const dirPattern = relativePath + '/';
|
if (!isIgnored(fullPath, true, active)) {
|
||||||
let excluded = false;
|
walk(fullPath, active);
|
||||||
for (const pattern of config.exclude) {
|
|
||||||
if (matchesGlob(dirPattern, pattern) || matchesGlob(relativePath, pattern)) {
|
|
||||||
excluded = true;
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if (!excluded) {
|
|
||||||
walk(fullPath);
|
|
||||||
}
|
}
|
||||||
} else if (entry.isFile()) {
|
} else if (entry.isFile()) {
|
||||||
if (shouldIncludeFile(relativePath, config)) {
|
if (!isIgnored(fullPath, false, active) && isSourceFile(relativePath)) {
|
||||||
files.push(relativePath);
|
files.push(relativePath);
|
||||||
count++;
|
count++;
|
||||||
onProgress?.(count, relativePath);
|
onProgress?.(count, relativePath);
|
||||||
@@ -430,7 +411,7 @@ function scanDirectoryWalk(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
walk(rootDir);
|
walk(rootDir, []);
|
||||||
return files;
|
return files;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -439,7 +420,6 @@ function scanDirectoryWalk(
|
|||||||
*/
|
*/
|
||||||
export class ExtractionOrchestrator {
|
export class ExtractionOrchestrator {
|
||||||
private rootDir: string;
|
private rootDir: string;
|
||||||
private config: CodeGraphConfig;
|
|
||||||
private queries: QueryBuilder;
|
private queries: QueryBuilder;
|
||||||
/**
|
/**
|
||||||
* Names of frameworks detected for this project, populated by indexAll().
|
* Names of frameworks detected for this project, populated by indexAll().
|
||||||
@@ -449,9 +429,8 @@ export class ExtractionOrchestrator {
|
|||||||
*/
|
*/
|
||||||
private detectedFrameworkNames: string[] | null = null;
|
private detectedFrameworkNames: string[] | null = null;
|
||||||
|
|
||||||
constructor(rootDir: string, config: CodeGraphConfig, queries: QueryBuilder) {
|
constructor(rootDir: string, queries: QueryBuilder) {
|
||||||
this.rootDir = rootDir;
|
this.rootDir = rootDir;
|
||||||
this.config = config;
|
|
||||||
this.queries = queries;
|
this.queries = queries;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -500,7 +479,7 @@ export class ExtractionOrchestrator {
|
|||||||
*/
|
*/
|
||||||
private ensureDetectedFrameworks(files?: string[]): string[] {
|
private ensureDetectedFrameworks(files?: string[]): string[] {
|
||||||
if (this.detectedFrameworkNames !== null) return this.detectedFrameworkNames;
|
if (this.detectedFrameworkNames !== null) return this.detectedFrameworkNames;
|
||||||
const fileList = files ?? scanDirectory(this.rootDir, this.config);
|
const fileList = files ?? scanDirectory(this.rootDir);
|
||||||
const context = this.buildDetectionContext(fileList);
|
const context = this.buildDetectionContext(fileList);
|
||||||
this.detectedFrameworkNames = detectFrameworks(context).map((r) => r.name);
|
this.detectedFrameworkNames = detectFrameworks(context).map((r) => r.name);
|
||||||
return this.detectedFrameworkNames;
|
return this.detectedFrameworkNames;
|
||||||
@@ -534,7 +513,7 @@ export class ExtractionOrchestrator {
|
|||||||
total: 0,
|
total: 0,
|
||||||
});
|
});
|
||||||
|
|
||||||
const files = await scanDirectoryAsync(this.rootDir, this.config, (current, file) => {
|
const files = await scanDirectoryAsync(this.rootDir, (current, file) => {
|
||||||
onProgress?.({
|
onProgress?.({
|
||||||
phase: 'scanning',
|
phase: 'scanning',
|
||||||
current,
|
current,
|
||||||
@@ -802,18 +781,16 @@ export class ExtractionOrchestrator {
|
|||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Honour config.maxFileSize. Without this check, vendored
|
// Honour MAX_FILE_SIZE. Without this check, vendored generated
|
||||||
// generated headers, minified bundles, and other multi-MB
|
// headers, minified bundles, and other multi-MB files get indexed,
|
||||||
// files get indexed despite the user setting a size cap —
|
// wasting WASM heap and the worker recycle budget on inputs with no
|
||||||
// wasting WASM heap and the worker recycle budget on inputs
|
// useful symbols. The single-file extractFile path already enforces
|
||||||
// the user explicitly opted out of. The single-file extractFile
|
// this; the bulk path used to silently skip the check.
|
||||||
// path already enforces this; the bulk path used to silently
|
if (stats.size > MAX_FILE_SIZE) {
|
||||||
// skip the check.
|
|
||||||
if (stats.size > this.config.maxFileSize) {
|
|
||||||
processed++;
|
processed++;
|
||||||
filesSkipped++;
|
filesSkipped++;
|
||||||
errors.push({
|
errors.push({
|
||||||
message: `File exceeds max size (${stats.size} > ${this.config.maxFileSize})`,
|
message: `File exceeds max size (${stats.size} > ${MAX_FILE_SIZE})`,
|
||||||
filePath,
|
filePath,
|
||||||
severity: 'warning',
|
severity: 'warning',
|
||||||
code: 'size_exceeded',
|
code: 'size_exceeded',
|
||||||
@@ -1108,14 +1085,14 @@ export class ExtractionOrchestrator {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Check file size
|
// Check file size
|
||||||
if (stats.size > this.config.maxFileSize) {
|
if (stats.size > MAX_FILE_SIZE) {
|
||||||
return {
|
return {
|
||||||
nodes: [],
|
nodes: [],
|
||||||
edges: [],
|
edges: [],
|
||||||
unresolvedReferences: [],
|
unresolvedReferences: [],
|
||||||
errors: [
|
errors: [
|
||||||
{
|
{
|
||||||
message: `File exceeds max size (${stats.size} > ${this.config.maxFileSize})`,
|
message: `File exceeds max size (${stats.size} > ${MAX_FILE_SIZE})`,
|
||||||
filePath: relativePath,
|
filePath: relativePath,
|
||||||
severity: 'warning',
|
severity: 'warning',
|
||||||
code: 'size_exceeded',
|
code: 'size_exceeded',
|
||||||
@@ -1245,7 +1222,7 @@ export class ExtractionOrchestrator {
|
|||||||
});
|
});
|
||||||
|
|
||||||
const filesToIndex: string[] = [];
|
const filesToIndex: string[] = [];
|
||||||
const gitChanges = getGitChangedFiles(this.rootDir, this.config);
|
const gitChanges = getGitChangedFiles(this.rootDir);
|
||||||
|
|
||||||
if (gitChanges) {
|
if (gitChanges) {
|
||||||
// === Git fast path ===
|
// === Git fast path ===
|
||||||
@@ -1291,7 +1268,7 @@ export class ExtractionOrchestrator {
|
|||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
// === Fallback: full scan (non-git project or git failure) ===
|
// === Fallback: full scan (non-git project or git failure) ===
|
||||||
const currentFiles = new Set(scanDirectory(this.rootDir, this.config));
|
const currentFiles = new Set(scanDirectory(this.rootDir));
|
||||||
filesChecked = currentFiles.size;
|
filesChecked = currentFiles.size;
|
||||||
|
|
||||||
// Build Map for O(1) lookups instead of .find() per file
|
// Build Map for O(1) lookups instead of .find() per file
|
||||||
@@ -1376,7 +1353,7 @@ export class ExtractionOrchestrator {
|
|||||||
* Uses git status as a fast path when available, falling back to full scan.
|
* Uses git status as a fast path when available, falling back to full scan.
|
||||||
*/
|
*/
|
||||||
getChangedFiles(): { added: string[]; modified: string[]; removed: string[] } {
|
getChangedFiles(): { added: string[]; modified: string[]; removed: string[] } {
|
||||||
const gitChanges = getGitChangedFiles(this.rootDir, this.config);
|
const gitChanges = getGitChangedFiles(this.rootDir);
|
||||||
|
|
||||||
if (gitChanges) {
|
if (gitChanges) {
|
||||||
// === Git fast path ===
|
// === Git fast path ===
|
||||||
@@ -1420,7 +1397,7 @@ export class ExtractionOrchestrator {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// === Fallback: full scan (non-git project or git failure) ===
|
// === Fallback: full scan (non-git project or git failure) ===
|
||||||
const currentFiles = new Set(scanDirectory(this.rootDir, this.config));
|
const currentFiles = new Set(scanDirectory(this.rootDir));
|
||||||
const trackedFiles = this.queries.getAllFiles();
|
const trackedFiles = this.queries.getAllFiles();
|
||||||
|
|
||||||
// Build Map for O(1) lookups
|
// Build Map for O(1) lookups
|
||||||
@@ -1467,4 +1444,4 @@ export class ExtractionOrchestrator {
|
|||||||
|
|
||||||
// Re-export useful types and functions
|
// Re-export useful types and functions
|
||||||
export { extractFromSource } from './tree-sitter';
|
export { extractFromSource } from './tree-sitter';
|
||||||
export { detectLanguage, isLanguageSupported, isGrammarLoaded, getSupportedLanguages, initGrammars, loadGrammarsForLanguages, loadAllGrammars } from './grammars';
|
export { detectLanguage, isSourceFile, isLanguageSupported, isGrammarLoaded, getSupportedLanguages, initGrammars, loadGrammarsForLanguages, loadAllGrammars } from './grammars';
|
||||||
|
|||||||
+6
-62
@@ -7,7 +7,6 @@
|
|||||||
|
|
||||||
import * as path from 'path';
|
import * as path from 'path';
|
||||||
import {
|
import {
|
||||||
CodeGraphConfig,
|
|
||||||
Node,
|
Node,
|
||||||
Edge,
|
Edge,
|
||||||
FileRecord,
|
FileRecord,
|
||||||
@@ -25,7 +24,6 @@ import {
|
|||||||
} from './types';
|
} from './types';
|
||||||
import { DatabaseConnection, getDatabasePath } from './db';
|
import { DatabaseConnection, getDatabasePath } from './db';
|
||||||
import { QueryBuilder } from './db/queries';
|
import { QueryBuilder } from './db/queries';
|
||||||
import { loadConfig, saveConfig, createDefaultConfig } from './config';
|
|
||||||
import {
|
import {
|
||||||
isInitialized,
|
isInitialized,
|
||||||
createDirectory,
|
createDirectory,
|
||||||
@@ -53,7 +51,6 @@ import { FileWatcher, WatchOptions } from './sync';
|
|||||||
// Re-export types for consumers
|
// Re-export types for consumers
|
||||||
export * from './types';
|
export * from './types';
|
||||||
export { getDatabasePath } from './db';
|
export { getDatabasePath } from './db';
|
||||||
export { getConfigPath } from './config';
|
|
||||||
export {
|
export {
|
||||||
getCodeGraphDir,
|
getCodeGraphDir,
|
||||||
isInitialized,
|
isInitialized,
|
||||||
@@ -85,9 +82,6 @@ export { MCPServer } from './mcp';
|
|||||||
* Options for initializing a new CodeGraph project
|
* Options for initializing a new CodeGraph project
|
||||||
*/
|
*/
|
||||||
export interface InitOptions {
|
export interface InitOptions {
|
||||||
/** Custom configuration overrides */
|
|
||||||
config?: Partial<CodeGraphConfig>;
|
|
||||||
|
|
||||||
/** Whether to run initial indexing after init */
|
/** Whether to run initial indexing after init */
|
||||||
index?: boolean;
|
index?: boolean;
|
||||||
|
|
||||||
@@ -128,7 +122,6 @@ export interface IndexOptions {
|
|||||||
export class CodeGraph {
|
export class CodeGraph {
|
||||||
private db: DatabaseConnection;
|
private db: DatabaseConnection;
|
||||||
private queries: QueryBuilder;
|
private queries: QueryBuilder;
|
||||||
private config: CodeGraphConfig;
|
|
||||||
private projectRoot: string;
|
private projectRoot: string;
|
||||||
private orchestrator: ExtractionOrchestrator;
|
private orchestrator: ExtractionOrchestrator;
|
||||||
private resolver: ReferenceResolver;
|
private resolver: ReferenceResolver;
|
||||||
@@ -148,17 +141,15 @@ export class CodeGraph {
|
|||||||
private constructor(
|
private constructor(
|
||||||
db: DatabaseConnection,
|
db: DatabaseConnection,
|
||||||
queries: QueryBuilder,
|
queries: QueryBuilder,
|
||||||
config: CodeGraphConfig,
|
|
||||||
projectRoot: string
|
projectRoot: string
|
||||||
) {
|
) {
|
||||||
this.db = db;
|
this.db = db;
|
||||||
this.queries = queries;
|
this.queries = queries;
|
||||||
this.config = config;
|
|
||||||
this.projectRoot = projectRoot;
|
this.projectRoot = projectRoot;
|
||||||
this.fileLock = new FileLock(
|
this.fileLock = new FileLock(
|
||||||
path.join(projectRoot, '.codegraph', 'codegraph.lock')
|
path.join(projectRoot, '.codegraph', 'codegraph.lock')
|
||||||
);
|
);
|
||||||
this.orchestrator = new ExtractionOrchestrator(projectRoot, config, queries);
|
this.orchestrator = new ExtractionOrchestrator(projectRoot, queries);
|
||||||
this.resolver = createResolver(projectRoot, queries);
|
this.resolver = createResolver(projectRoot, queries);
|
||||||
this.graphManager = new GraphQueryManager(queries);
|
this.graphManager = new GraphQueryManager(queries);
|
||||||
this.traverser = new GraphTraverser(queries);
|
this.traverser = new GraphTraverser(queries);
|
||||||
@@ -194,19 +185,12 @@ export class CodeGraph {
|
|||||||
// Create directory structure
|
// Create directory structure
|
||||||
createDirectory(resolvedRoot);
|
createDirectory(resolvedRoot);
|
||||||
|
|
||||||
// Create and save configuration
|
|
||||||
const config = createDefaultConfig(resolvedRoot);
|
|
||||||
if (options.config) {
|
|
||||||
Object.assign(config, options.config);
|
|
||||||
}
|
|
||||||
saveConfig(resolvedRoot, config);
|
|
||||||
|
|
||||||
// Initialize database
|
// Initialize database
|
||||||
const dbPath = getDatabasePath(resolvedRoot);
|
const dbPath = getDatabasePath(resolvedRoot);
|
||||||
const db = DatabaseConnection.initialize(dbPath);
|
const db = DatabaseConnection.initialize(dbPath);
|
||||||
const queries = new QueryBuilder(db.getDb());
|
const queries = new QueryBuilder(db.getDb());
|
||||||
|
|
||||||
const instance = new CodeGraph(db, queries, config, resolvedRoot);
|
const instance = new CodeGraph(db, queries, resolvedRoot);
|
||||||
|
|
||||||
// Run initial indexing if requested
|
// Run initial indexing if requested
|
||||||
if (options.index) {
|
if (options.index) {
|
||||||
@@ -219,7 +203,7 @@ export class CodeGraph {
|
|||||||
/**
|
/**
|
||||||
* Initialize synchronously (without indexing)
|
* Initialize synchronously (without indexing)
|
||||||
*/
|
*/
|
||||||
static initSync(projectRoot: string, options: Omit<InitOptions, 'index' | 'onProgress'> = {}): CodeGraph {
|
static initSync(projectRoot: string): CodeGraph {
|
||||||
const resolvedRoot = path.resolve(projectRoot);
|
const resolvedRoot = path.resolve(projectRoot);
|
||||||
|
|
||||||
// Check if already initialized
|
// Check if already initialized
|
||||||
@@ -230,19 +214,12 @@ export class CodeGraph {
|
|||||||
// Create directory structure
|
// Create directory structure
|
||||||
createDirectory(resolvedRoot);
|
createDirectory(resolvedRoot);
|
||||||
|
|
||||||
// Create and save configuration
|
|
||||||
const config = createDefaultConfig(resolvedRoot);
|
|
||||||
if (options.config) {
|
|
||||||
Object.assign(config, options.config);
|
|
||||||
}
|
|
||||||
saveConfig(resolvedRoot, config);
|
|
||||||
|
|
||||||
// Initialize database
|
// Initialize database
|
||||||
const dbPath = getDatabasePath(resolvedRoot);
|
const dbPath = getDatabasePath(resolvedRoot);
|
||||||
const db = DatabaseConnection.initialize(dbPath);
|
const db = DatabaseConnection.initialize(dbPath);
|
||||||
const queries = new QueryBuilder(db.getDb());
|
const queries = new QueryBuilder(db.getDb());
|
||||||
|
|
||||||
return new CodeGraph(db, queries, config, resolvedRoot);
|
return new CodeGraph(db, queries, resolvedRoot);
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -267,15 +244,12 @@ export class CodeGraph {
|
|||||||
throw new Error(`Invalid CodeGraph directory: ${validation.errors.join(', ')}`);
|
throw new Error(`Invalid CodeGraph directory: ${validation.errors.join(', ')}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Load configuration
|
|
||||||
const config = loadConfig(resolvedRoot);
|
|
||||||
|
|
||||||
// Open database
|
// Open database
|
||||||
const dbPath = getDatabasePath(resolvedRoot);
|
const dbPath = getDatabasePath(resolvedRoot);
|
||||||
const db = DatabaseConnection.open(dbPath);
|
const db = DatabaseConnection.open(dbPath);
|
||||||
const queries = new QueryBuilder(db.getDb());
|
const queries = new QueryBuilder(db.getDb());
|
||||||
|
|
||||||
const instance = new CodeGraph(db, queries, config, resolvedRoot);
|
const instance = new CodeGraph(db, queries, resolvedRoot);
|
||||||
|
|
||||||
// Sync if requested
|
// Sync if requested
|
||||||
if (options.sync) {
|
if (options.sync) {
|
||||||
@@ -302,15 +276,12 @@ export class CodeGraph {
|
|||||||
throw new Error(`Invalid CodeGraph directory: ${validation.errors.join(', ')}`);
|
throw new Error(`Invalid CodeGraph directory: ${validation.errors.join(', ')}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Load configuration
|
|
||||||
const config = loadConfig(resolvedRoot);
|
|
||||||
|
|
||||||
// Open database
|
// Open database
|
||||||
const dbPath = getDatabasePath(resolvedRoot);
|
const dbPath = getDatabasePath(resolvedRoot);
|
||||||
const db = DatabaseConnection.open(dbPath);
|
const db = DatabaseConnection.open(dbPath);
|
||||||
const queries = new QueryBuilder(db.getDb());
|
const queries = new QueryBuilder(db.getDb());
|
||||||
|
|
||||||
return new CodeGraph(db, queries, config, resolvedRoot);
|
return new CodeGraph(db, queries, resolvedRoot);
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -330,32 +301,6 @@ export class CodeGraph {
|
|||||||
this.db.close();
|
this.db.close();
|
||||||
}
|
}
|
||||||
|
|
||||||
// ===========================================================================
|
|
||||||
// Configuration
|
|
||||||
// ===========================================================================
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get the current configuration
|
|
||||||
*/
|
|
||||||
getConfig(): CodeGraphConfig {
|
|
||||||
return { ...this.config };
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Update configuration
|
|
||||||
*/
|
|
||||||
updateConfig(updates: Partial<CodeGraphConfig>): void {
|
|
||||||
Object.assign(this.config, updates);
|
|
||||||
saveConfig(this.projectRoot, this.config);
|
|
||||||
// Recreate orchestrator and resolver with new config
|
|
||||||
this.orchestrator = new ExtractionOrchestrator(
|
|
||||||
this.projectRoot,
|
|
||||||
this.config,
|
|
||||||
this.queries
|
|
||||||
);
|
|
||||||
this.resolver = createResolver(this.projectRoot, this.queries);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Get the project root directory
|
* Get the project root directory
|
||||||
*/
|
*/
|
||||||
@@ -515,7 +460,6 @@ export class CodeGraph {
|
|||||||
|
|
||||||
this.watcher = new FileWatcher(
|
this.watcher = new FileWatcher(
|
||||||
this.projectRoot,
|
this.projectRoot,
|
||||||
this.config,
|
|
||||||
async () => {
|
async () => {
|
||||||
const result = await this.sync();
|
const result = await this.sync();
|
||||||
const filesChanged = result.filesAdded + result.filesModified + result.filesRemoved;
|
const filesChanged = result.filesAdded + result.filesModified + result.filesRemoved;
|
||||||
|
|||||||
+4
-8
@@ -9,8 +9,7 @@
|
|||||||
*/
|
*/
|
||||||
|
|
||||||
import * as fs from 'fs';
|
import * as fs from 'fs';
|
||||||
import { CodeGraphConfig } from '../types';
|
import { isSourceFile } from '../extraction';
|
||||||
import { shouldIncludeFile } from '../extraction';
|
|
||||||
import { logDebug, logWarn } from '../errors';
|
import { logDebug, logWarn } from '../errors';
|
||||||
import { normalizePath } from '../utils';
|
import { normalizePath } from '../utils';
|
||||||
import { watchDisabledReason } from './watch-policy';
|
import { watchDisabledReason } from './watch-policy';
|
||||||
@@ -44,7 +43,7 @@ export interface WatchOptions {
|
|||||||
* Design goals:
|
* Design goals:
|
||||||
* - Minimal resource usage (native OS file events, no polling)
|
* - Minimal resource usage (native OS file events, no polling)
|
||||||
* - Debounced to avoid thrashing on rapid saves
|
* - Debounced to avoid thrashing on rapid saves
|
||||||
* - Filters against CodeGraph include/exclude patterns
|
* - Filters to supported source files by extension
|
||||||
* - Ignores .codegraph/ directory changes
|
* - Ignores .codegraph/ directory changes
|
||||||
*/
|
*/
|
||||||
export class FileWatcher {
|
export class FileWatcher {
|
||||||
@@ -55,7 +54,6 @@ export class FileWatcher {
|
|||||||
private stopped = false;
|
private stopped = false;
|
||||||
|
|
||||||
private readonly projectRoot: string;
|
private readonly projectRoot: string;
|
||||||
private readonly config: CodeGraphConfig;
|
|
||||||
private readonly debounceMs: number;
|
private readonly debounceMs: number;
|
||||||
private readonly syncFn: () => Promise<{ filesChanged: number; durationMs: number }>;
|
private readonly syncFn: () => Promise<{ filesChanged: number; durationMs: number }>;
|
||||||
private readonly onSyncComplete?: WatchOptions['onSyncComplete'];
|
private readonly onSyncComplete?: WatchOptions['onSyncComplete'];
|
||||||
@@ -63,12 +61,10 @@ export class FileWatcher {
|
|||||||
|
|
||||||
constructor(
|
constructor(
|
||||||
projectRoot: string,
|
projectRoot: string,
|
||||||
config: CodeGraphConfig,
|
|
||||||
syncFn: () => Promise<{ filesChanged: number; durationMs: number }>,
|
syncFn: () => Promise<{ filesChanged: number; durationMs: number }>,
|
||||||
options: WatchOptions = {}
|
options: WatchOptions = {}
|
||||||
) {
|
) {
|
||||||
this.projectRoot = projectRoot;
|
this.projectRoot = projectRoot;
|
||||||
this.config = config;
|
|
||||||
this.syncFn = syncFn;
|
this.syncFn = syncFn;
|
||||||
this.debounceMs = options.debounceMs ?? 2000;
|
this.debounceMs = options.debounceMs ?? 2000;
|
||||||
this.onSyncComplete = options.onSyncComplete;
|
this.onSyncComplete = options.onSyncComplete;
|
||||||
@@ -112,8 +108,8 @@ export class FileWatcher {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Filter against include/exclude patterns
|
// Only sync changes to files we can actually parse.
|
||||||
if (!shouldIncludeFile(normalized, this.config)) {
|
if (!isSourceFile(normalized)) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
-291
@@ -426,297 +426,6 @@ export interface CodeBlock {
|
|||||||
node?: Node;
|
node?: Node;
|
||||||
}
|
}
|
||||||
|
|
||||||
// =============================================================================
|
|
||||||
// Configuration Types
|
|
||||||
// =============================================================================
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Framework-specific hints for better extraction
|
|
||||||
*/
|
|
||||||
export interface FrameworkHint {
|
|
||||||
/** Framework name (react, express, django, etc.) */
|
|
||||||
name: string;
|
|
||||||
|
|
||||||
/** Version constraint if relevant */
|
|
||||||
version?: string;
|
|
||||||
|
|
||||||
/** Custom patterns for this framework */
|
|
||||||
patterns?: {
|
|
||||||
/** Component detection patterns */
|
|
||||||
components?: string[];
|
|
||||||
/** Route detection patterns */
|
|
||||||
routes?: string[];
|
|
||||||
/** Model detection patterns */
|
|
||||||
models?: string[];
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Configuration for a CodeGraph project
|
|
||||||
*/
|
|
||||||
export interface CodeGraphConfig {
|
|
||||||
/** Schema version for migrations */
|
|
||||||
version: number;
|
|
||||||
|
|
||||||
/** Root directory of the project */
|
|
||||||
rootDir: string;
|
|
||||||
|
|
||||||
/** Glob patterns for files to include */
|
|
||||||
include: string[];
|
|
||||||
|
|
||||||
/** Glob patterns for files to exclude */
|
|
||||||
exclude: string[];
|
|
||||||
|
|
||||||
/** Languages to process (auto-detected if empty) */
|
|
||||||
languages: Language[];
|
|
||||||
|
|
||||||
/** Framework hints for better extraction */
|
|
||||||
frameworks: FrameworkHint[];
|
|
||||||
|
|
||||||
/** Maximum file size to process (in bytes) */
|
|
||||||
maxFileSize: number;
|
|
||||||
|
|
||||||
/** Whether to extract docstrings */
|
|
||||||
extractDocstrings: boolean;
|
|
||||||
|
|
||||||
/** Whether to track call sites */
|
|
||||||
trackCallSites: boolean;
|
|
||||||
|
|
||||||
/** Custom symbol patterns to extract */
|
|
||||||
customPatterns?: {
|
|
||||||
/** Name for this pattern group */
|
|
||||||
name: string;
|
|
||||||
/** Regex pattern to match */
|
|
||||||
pattern: string;
|
|
||||||
/** Node kind to assign */
|
|
||||||
kind: NodeKind;
|
|
||||||
}[];
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Default configuration values
|
|
||||||
*/
|
|
||||||
export const DEFAULT_CONFIG: CodeGraphConfig = {
|
|
||||||
version: 1,
|
|
||||||
rootDir: '.',
|
|
||||||
include: [
|
|
||||||
// TypeScript/JavaScript
|
|
||||||
'**/*.ts',
|
|
||||||
'**/*.tsx',
|
|
||||||
'**/*.js',
|
|
||||||
'**/*.jsx',
|
|
||||||
// Python
|
|
||||||
'**/*.py',
|
|
||||||
// Go
|
|
||||||
'**/*.go',
|
|
||||||
// Rust
|
|
||||||
'**/*.rs',
|
|
||||||
// Java
|
|
||||||
'**/*.java',
|
|
||||||
// C/C++
|
|
||||||
'**/*.c',
|
|
||||||
'**/*.h',
|
|
||||||
'**/*.cpp',
|
|
||||||
'**/*.hpp',
|
|
||||||
'**/*.cc',
|
|
||||||
'**/*.cxx',
|
|
||||||
// C#
|
|
||||||
'**/*.cs',
|
|
||||||
// PHP
|
|
||||||
'**/*.php',
|
|
||||||
// Drupal-specific PHP extensions
|
|
||||||
'**/*.module',
|
|
||||||
'**/*.install',
|
|
||||||
'**/*.theme',
|
|
||||||
'**/*.inc',
|
|
||||||
// Drupal routing YAML
|
|
||||||
'**/*.routing.yml',
|
|
||||||
// Twig templates
|
|
||||||
'**/*.twig',
|
|
||||||
// Ruby
|
|
||||||
'**/*.rb',
|
|
||||||
// Swift
|
|
||||||
'**/*.swift',
|
|
||||||
// Kotlin
|
|
||||||
'**/*.kt',
|
|
||||||
'**/*.kts',
|
|
||||||
// Dart
|
|
||||||
'**/*.dart',
|
|
||||||
// Svelte
|
|
||||||
'**/*.svelte',
|
|
||||||
// Vue
|
|
||||||
'**/*.vue',
|
|
||||||
// Liquid (Shopify themes)
|
|
||||||
'**/*.liquid',
|
|
||||||
// Pascal / Delphi
|
|
||||||
'**/*.pas',
|
|
||||||
'**/*.dpr',
|
|
||||||
'**/*.dpk',
|
|
||||||
'**/*.lpr',
|
|
||||||
'**/*.dfm',
|
|
||||||
'**/*.fmx',
|
|
||||||
// Scala
|
|
||||||
'**/*.scala',
|
|
||||||
'**/*.sc',
|
|
||||||
// Lua
|
|
||||||
'**/*.lua',
|
|
||||||
// Luau
|
|
||||||
'**/*.luau',
|
|
||||||
],
|
|
||||||
exclude: [
|
|
||||||
// Version control
|
|
||||||
'**/.git/**',
|
|
||||||
|
|
||||||
// Dependencies
|
|
||||||
'**/node_modules/**',
|
|
||||||
'**/vendor/**',
|
|
||||||
'**/Pods/**',
|
|
||||||
|
|
||||||
// Generic build outputs
|
|
||||||
'**/dist/**',
|
|
||||||
'**/build/**',
|
|
||||||
'**/out/**',
|
|
||||||
'**/bin/**',
|
|
||||||
'**/obj/**',
|
|
||||||
'**/target/**',
|
|
||||||
|
|
||||||
// JavaScript/TypeScript
|
|
||||||
'**/*.min.js',
|
|
||||||
'**/*.bundle.js',
|
|
||||||
'**/.next/**',
|
|
||||||
'**/.nuxt/**',
|
|
||||||
'**/.svelte-kit/**',
|
|
||||||
'**/.output/**',
|
|
||||||
'**/.turbo/**',
|
|
||||||
'**/.cache/**',
|
|
||||||
'**/.parcel-cache/**',
|
|
||||||
'**/.vite/**',
|
|
||||||
'**/.astro/**',
|
|
||||||
'**/.docusaurus/**',
|
|
||||||
'**/.gatsby/**',
|
|
||||||
'**/.webpack/**',
|
|
||||||
'**/.nx/**',
|
|
||||||
'**/.yarn/cache/**',
|
|
||||||
'**/.pnpm-store/**',
|
|
||||||
'**/storybook-static/**',
|
|
||||||
|
|
||||||
// React Native / Expo
|
|
||||||
'**/.expo/**',
|
|
||||||
'**/web-build/**',
|
|
||||||
'**/ios/Pods/**',
|
|
||||||
'**/ios/build/**',
|
|
||||||
'**/android/build/**',
|
|
||||||
'**/android/.gradle/**',
|
|
||||||
|
|
||||||
// Python
|
|
||||||
'**/__pycache__/**',
|
|
||||||
'**/.venv/**',
|
|
||||||
'**/venv/**',
|
|
||||||
'**/site-packages/**',
|
|
||||||
'**/dist-packages/**',
|
|
||||||
'**/.pytest_cache/**',
|
|
||||||
'**/.mypy_cache/**',
|
|
||||||
'**/.ruff_cache/**',
|
|
||||||
'**/.tox/**',
|
|
||||||
'**/.nox/**',
|
|
||||||
'**/*.egg-info/**',
|
|
||||||
'**/.eggs/**',
|
|
||||||
|
|
||||||
// Go
|
|
||||||
'**/go/pkg/mod/**',
|
|
||||||
|
|
||||||
// Rust
|
|
||||||
'**/target/debug/**',
|
|
||||||
'**/target/release/**',
|
|
||||||
|
|
||||||
// Java/Kotlin/Gradle
|
|
||||||
'**/.gradle/**',
|
|
||||||
'**/.m2/**',
|
|
||||||
'**/generated-sources/**',
|
|
||||||
'**/.kotlin/**',
|
|
||||||
|
|
||||||
// Dart/Flutter
|
|
||||||
'**/.dart_tool/**',
|
|
||||||
|
|
||||||
// C#/.NET
|
|
||||||
'**/.vs/**',
|
|
||||||
'**/.nuget/**',
|
|
||||||
'**/artifacts/**',
|
|
||||||
'**/publish/**',
|
|
||||||
|
|
||||||
// C/C++
|
|
||||||
'**/cmake-build-*/**',
|
|
||||||
'**/CMakeFiles/**',
|
|
||||||
'**/bazel-*/**',
|
|
||||||
'**/vcpkg_installed/**',
|
|
||||||
'**/.conan/**',
|
|
||||||
'**/Debug/**',
|
|
||||||
'**/Release/**',
|
|
||||||
'**/x64/**',
|
|
||||||
'**/.pio/**', // Platform.io (IoT/embedded build artifacts and library deps)
|
|
||||||
|
|
||||||
// Electron
|
|
||||||
'**/release/**',
|
|
||||||
'**/*.app/**',
|
|
||||||
'**/*.asar',
|
|
||||||
|
|
||||||
// Swift/iOS/Xcode
|
|
||||||
'**/DerivedData/**',
|
|
||||||
'**/.build/**',
|
|
||||||
'**/.swiftpm/**',
|
|
||||||
'**/xcuserdata/**',
|
|
||||||
'**/Carthage/Build/**',
|
|
||||||
'**/SourcePackages/**',
|
|
||||||
|
|
||||||
// Delphi/Pascal
|
|
||||||
'**/__history/**',
|
|
||||||
'**/__recovery/**',
|
|
||||||
'**/*.dcu',
|
|
||||||
|
|
||||||
// PHP
|
|
||||||
'**/.composer/**',
|
|
||||||
'**/storage/framework/**',
|
|
||||||
'**/bootstrap/cache/**',
|
|
||||||
|
|
||||||
// Drupal - core and contrib are rarely customised; index only custom code
|
|
||||||
'**/web/core/**',
|
|
||||||
'**/web/modules/contrib/**',
|
|
||||||
'**/web/themes/contrib/**',
|
|
||||||
|
|
||||||
// Ruby
|
|
||||||
'**/.bundle/**',
|
|
||||||
'**/tmp/cache/**',
|
|
||||||
'**/public/assets/**',
|
|
||||||
'**/public/packs/**',
|
|
||||||
'**/.yardoc/**',
|
|
||||||
|
|
||||||
// Testing/Coverage
|
|
||||||
'**/coverage/**',
|
|
||||||
'**/htmlcov/**',
|
|
||||||
'**/.nyc_output/**',
|
|
||||||
'**/test-results/**',
|
|
||||||
'**/.coverage/**',
|
|
||||||
|
|
||||||
// IDE/Editor
|
|
||||||
'**/.idea/**',
|
|
||||||
|
|
||||||
// Logs and temp
|
|
||||||
'**/logs/**',
|
|
||||||
'**/tmp/**',
|
|
||||||
'**/temp/**',
|
|
||||||
|
|
||||||
// Documentation build output
|
|
||||||
'**/_build/**',
|
|
||||||
'**/docs/_build/**',
|
|
||||||
'**/site/**',
|
|
||||||
],
|
|
||||||
languages: [],
|
|
||||||
frameworks: [],
|
|
||||||
maxFileSize: 1024 * 1024, // 1MB
|
|
||||||
extractDocstrings: true,
|
|
||||||
trackCallSites: true,
|
|
||||||
};
|
|
||||||
|
|
||||||
// =============================================================================
|
// =============================================================================
|
||||||
// Database Types
|
// Database Types
|
||||||
// =============================================================================
|
// =============================================================================
|
||||||
|
|||||||
Reference in New Issue
Block a user