feat(offload): reasoning offload for codegraph_explore (bring-your-own endpoint)
codegraph_explore can now hand the source it retrieved to a reasoning model you point at — any OpenAI-compatible endpoint (Cerebras, OpenAI, a local vLLM/Ollama) with your own key — and return that model's tight, cited answer instead of the raw source dump. The agent's main context gets the answer in far fewer tokens, at the cost of one network round-trip. Off by default. Configure with `codegraph offload set-endpoint <url> --model <m> --key-env <ENV>` (or the CODEGRAPH_OFFLOAD_* env vars); status/disable manage it. The API key is never written to disk — the config stores the NAME of an env var and the key is read from it at call time. Strictly degradable: any failure (no endpoint, network, timeout, empty answer) returns null and the call falls back to the local source, so the offload can never surface an error to the agent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
f34f606342
commit
db4c9f3641
@@ -29,6 +29,7 @@ import {
|
||||
import { clamp, validatePathWithinRoot, validateProjectPath, isConfigLeafNode, CONFIG_LEAF_LANGUAGES } from '../utils';
|
||||
import { isGeneratedFile } from '../extraction/generated-detection';
|
||||
import { scanDynamicDispatch } from './dynamic-boundaries';
|
||||
import { isOffloadEnabled, synthesizeOffload } from '../reasoning/reasoner';
|
||||
|
||||
/**
|
||||
* An expected, recoverable "codegraph can't serve this" condition — most
|
||||
@@ -2960,6 +2961,17 @@ export class ToolHandler {
|
||||
// necessary overflow above the 24K budget, but hard-stop at 25K — never into
|
||||
// externalize territory.
|
||||
const output = flow.text + lines.join('\n');
|
||||
|
||||
// Reasoning offload (opt-in, bring-your-own endpoint): when configured, hand
|
||||
// the assembled source + the query to a reasoning model and return its
|
||||
// synthesized answer instead of the raw source dump. Reasons over the FULL
|
||||
// assembled context (pre-truncation). Strictly degradable — any failure
|
||||
// returns null and we fall through to returning the local source below.
|
||||
if (isOffloadEnabled()) {
|
||||
const synthesized = await synthesizeOffload({ query, context: output });
|
||||
if (synthesized) return this.textResult(synthesized);
|
||||
}
|
||||
|
||||
const hardCeiling = Math.min(Math.round(budget.maxOutputChars * 1.5), 25000);
|
||||
if (output.length > hardCeiling) {
|
||||
// Cut at a FILE-SECTION boundary (the last `#### ` header before the
|
||||
|
||||
Reference in New Issue
Block a user