github/codegraph
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(offload): reasoning offload for codegraph_explore (bring-your-own endpoint)

Browse Source 
codegraph_explore can now hand the source it retrieved to a reasoning model you
point at — any OpenAI-compatible endpoint (Cerebras, OpenAI, a local vLLM/Ollama)
with your own key — and return that model's tight, cited answer instead of the
raw source dump. The agent's main context gets the answer in far fewer tokens, at
the cost of one network round-trip.

Off by default. Configure with `codegraph offload set-endpoint <url> --model <m>
--key-env <ENV>` (or the CODEGRAPH_OFFLOAD_* env vars); status/disable manage it.
The API key is never written to disk — the config stores the NAME of an env var
and the key is read from it at call time. Strictly degradable: any failure
(no endpoint, network, timeout, empty answer) returns null and the call falls
back to the local source, so the offload can never surface an error to the agent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Colby McHenry
2026-06-17 14:18:22 -05:00
co-authored by Claude Opus 4.8
parent f34f606342
commit db4c9f3641
7 changed files with 619 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/bin/codegraph.ts
+61
View File
@@ -36,6 +36,7 @@ import { installFatalHandlers } from './fatal-handler';
import { relaunchWithWasmRuntimeFlagsIfNeeded } from '../extraction/wasm-runtime-flags';
import { EXTRACTION_VERSION } from '../extraction/extraction-version';
import { getTelemetry, TELEMETRY_DOCS, recordIndexEvent } from '../telemetry';
import { writeOffloadConfig, resolveOffload } from '../reasoning/config';
// Lazy-load heavy modules (CodeGraph, runInstaller) to keep CLI startup fast.
async function loadCodeGraph(): Promise<typeof import('../index')> {
@@ -1348,6 +1349,66 @@ program
});
});
/**
* codegraph offload — configure the reasoning offload (bring-your-own endpoint).
*
* When set, codegraph_explore reasons over its assembled source with a remote
* model and returns the synthesized answer instead of the raw source dump.
*/
const offloadCmd = program
.command('offload')
.description('Configure the reasoning offload — let codegraph_explore answer via your own reasoning model');
offloadCmd
.command('set-endpoint <url>')
.description('Send explore output to an OpenAI-compatible reasoning endpoint (URL ends in /v1)')
.option('--model <model>', 'Model id to request', 'gpt-oss-120b')
.option('--key-env <ENV>', 'Name of the env var holding the API key (the key is never written to disk)')
.option('--effort <effort>', 'reasoning_effort: low | medium | high')
.option('--style <style>', 'Output style: plain | report')
.action((url: string, opts: { model?: string; keyEnv?: string; effort?: string; style?: string }) => {
writeOffloadConfig({
url,
model: opts.model,
keyEnv: opts.keyEnv,
effort: opts.effort,
style: opts.style,
});
success(`Reasoning offload enabled → ${url}`);
info(` model: ${opts.model || 'gpt-oss-120b'}`);
if (opts.keyEnv) info(` key: read from $${opts.keyEnv} at call time`);
else warn(' no API key configured — pass --key-env <ENV> (or set CODEGRAPH_OFFLOAD_KEY) if your endpoint needs auth.');
info(' Restart your editor/agent session for running MCP servers to pick it up.');
});
offloadCmd
.command('status')
.description('Show the current reasoning-offload configuration')
.action(() => {
const c = resolveOffload();
if (!c.enabled) {
info('Reasoning offload: off. Enable with `codegraph offload set-endpoint <url>`.');
return;
}
success(`Reasoning offload: on (${c.origin === 'env' ? 'from environment' : 'configured'})`);
info(` endpoint: ${c.url}`);
info(` model: ${c.model}`);
info(` key: ${c.apiKey ? `present (from $${c.keySource})` : 'none'}`);
info(` effort: ${c.effort} style: ${c.style}`);
if (!c.apiKey) warn(' no API key resolved — set --key-env <ENV> or CODEGRAPH_OFFLOAD_KEY if your endpoint requires auth.');
});
offloadCmd
.command('disable')
.description('Turn off the reasoning offload')
.action(() => {
writeOffloadConfig(null);
success('Reasoning offload disabled.');
if (process.env.CODEGRAPH_OFFLOAD_URL) {
warn('Note: CODEGRAPH_OFFLOAD_URL is still set in your environment, which keeps it on. Unset it to fully disable.');
}
});
/**
* codegraph serve
*/