From cc388a14e8c27a680999197e17fea44b2b8600ae Mon Sep 17 00:00:00 2001 From: DawidWraga Date: Tue, 18 Aug 2026 18:14:12 +0100 Subject: [PATCH] feat(open-agents): make conversation history automatic --- .changeset/swift-agents-remember.md | 5 + packages/init/src/skills/explore.md | 43 +- packages/init/src/skills/fast-edit.md | 37 +- packages/open-agents/README.md | 61 +- .../__tests__/compact-background.test.ts | 110 +++ .../__tests__/compact-messages.test.ts | 92 --- .../open-agents/__tests__/compact.test.ts | 147 ---- .../open-agents/__tests__/history.test.ts | 248 +++++++ packages/open-agents/package.json | 3 +- packages/open-agents/src/cli.ts | 653 ++++++++++++------ packages/open-agents/src/core/compact.ts | 387 ----------- packages/open-agents/src/core/history.ts | 310 +++++++++ packages/open-agents/src/core/jobs.ts | 62 +- .../src/profiles/history-curator.ts | 17 + .../open-agents/src/profiles/spec-writer.ts | 22 - pnpm-lock.yaml | 3 - skills/explore/SKILL.md | 43 +- skills/fast-edit/SKILL.md | 37 +- 18 files changed, 1282 insertions(+), 998 deletions(-) create mode 100644 .changeset/swift-agents-remember.md create mode 100644 packages/open-agents/__tests__/compact-background.test.ts delete mode 100644 packages/open-agents/__tests__/compact-messages.test.ts delete mode 100644 packages/open-agents/__tests__/compact.test.ts create mode 100644 packages/open-agents/__tests__/history.test.ts delete mode 100644 packages/open-agents/src/core/compact.ts create mode 100644 packages/open-agents/src/core/history.ts create mode 100644 packages/open-agents/src/profiles/history-curator.ts delete mode 100644 packages/open-agents/src/profiles/spec-writer.ts diff --git a/.changeset/swift-agents-remember.md b/.changeset/swift-agents-remember.md new file mode 100644 index 0000000..a3eade9 --- /dev/null +++ b/.changeset/swift-agents-remember.md @@ -0,0 +1,5 @@ +--- +"@davstack/open-agents": minor +--- + +Include exact current-session conversation history automatically, with task-specific curation for oversized transcripts and a `--no-history` opt-out. diff --git a/packages/init/src/skills/explore.md b/packages/init/src/skills/explore.md index f12a04a..f810619 100644 --- a/packages/init/src/skills/explore.md +++ b/packages/init/src/skills/explore.md @@ -10,32 +10,43 @@ description: >- -Scope tightly, then run (backgrounded — the harness notifies you): +Default to one short `--task`, especially for lookups, traces, and other simple +requests. Use the user's request verbatim when it is already concise: - explore submit --file ~/.davstack/specs/.md +explore submit --task "Find where Juno's realtime voice is locked to marin, with exact path:line citations" -When the current conversation already contains the real context, prefer compact -mode over writing a spec: +Do not create a spec file or invent ``, ``, or `` merely +to wrap a task the user already stated. Conversation history supplies the +surrounding context automatically. - explore submit --compact-mode "audit supervisor handoffs" +Run one foreground submission with a long shell timeout. If the shell yields a +running handle, wait on that same handle; completion wakes the agent. Never pass +`--background`, poll `result`, or resubmit. -Keep the inline compact prompt to roughly 5-10 words. Do not paste details, -requirements, quotes, or file lists into it; the compact spec-writer reads the -current transcript/history and distills that context for the executor. Trust the -subagent handoff unless the task is genuinely too ambiguous from conversation -history. +Conversation history is included automatically. Keep the task to roughly one +sentence; do not repeat details, quotes, or file lists already present in the +conversation. History up to the direct budget goes to the executor unchanged; +oversized history is reduced to task-specific context first. Use `--no-history` +only when the conversation is irrelevant. `--compact-mode` remains a legacy +alias and is no longer needed. -For a **single scoped fact**, skip the spec file — inline it (no boilerplate): +If the conversation contains secrets, credentials, audit artifacts, or +user/health data, use `--no-history` and provide a sanitized scoped spec. - explore submit 'Exact signature + return type of resolve_query_adapter backend/src/query/adapter.py only' +For a single scoped fact, use `--task` with no boilerplate: -Many `--file` run in parallel from one command. Read the `result → ` -file for the answer. + explore submit --task "Find the exact signature and return type of resolve_query_adapter in backend/src/query/adapter.py" -Begin every `--file` spec with a markdown `# 3-5 word title` line — a short +Use `--spec-file` only when essential instructions not present in conversation +cannot fit cleanly in one sentence, or when supplying an intentionally prepared +multi-part spec. Never generate one pre-emptively. Multiple `--task` or +`--spec-file` inputs can run in parallel. + +If a spec file is genuinely needed, begin it with a markdown `# 3-5 word title` +line — a short overview of the task. The TUI agent viewer renders this as the job label; without it the viewer falls back to the first 5 words of the spec, which is -rarely meaningful. (Inline single-fact submits can skip the heading.) +rarely meaningful. The spec is just goal / context (the one gotcha) / scope tags. Do NOT add an output section — the structured `path:line` deliverable is automatic. diff --git a/packages/init/src/skills/fast-edit.md b/packages/init/src/skills/fast-edit.md index f63e46c..aa10f79 100644 --- a/packages/init/src/skills/fast-edit.md +++ b/packages/init/src/skills/fast-edit.md @@ -11,28 +11,41 @@ description: >- -Run (backgrounded — the harness notifies you): +Default to one short `--task` whenever the mechanical edit can be stated in one +sentence, especially when conversation history already contains the details: - npx fast-edit submit --file ~/.davstack/specs/.md + fast-edit submit --task "Rename fooBar to computeFoo and update its callers without changing behavior" -When the current conversation already contains the real context, prefer compact -mode over writing a spec: +Do not create a spec file or invent intent/changes/constraints tags merely to +wrap a task the user already stated. Conversation history supplies the +surrounding context automatically. - npx fast-edit submit --compact-mode "rename legacy adapter" +Run one foreground submission with a long shell timeout. If the shell yields a +running handle, wait on that same handle; completion wakes the agent. Never pass +`--background`, poll `result`, or resubmit. -Keep the inline compact prompt to roughly 5-10 words. Do not paste details, -requirements, quotes, or file lists into it; the compact spec-writer reads the -current transcript/history and distills that context for the executor. Trust the -subagent handoff unless the edit is too risky or underspecified from -conversation history. +Conversation history is included automatically. Keep the task to roughly one +sentence; do not repeat details, quotes, or file lists already present in the +conversation. History up to the direct budget goes to the executor unchanged; +oversized history is reduced to task-specific context first. Use `--no-history` +only when the conversation is irrelevant. `--compact-mode` remains a legacy +alias and is no longer needed. -**Routing test.** Delegate when a *short* intent+constraints spec is enough +If the conversation contains secrets, credentials, audit artifacts, or +user/health data, use `--no-history` and provide a sanitized scoped spec. + +**Routing test.** Delegate when a short task is enough for the executor to produce the **full** intended edit. If writing the spec would mean pasting the new file contents or spelling out every line, the spec costs as much as the edit — just do it yourself. (Having read the files is fine; verbatim-detail specs are the only real waste.) -Begin every spec with a markdown `# 3-5 word title` line — a short overview of +Use `--spec-file` only when essential constraints not present in conversation +cannot fit cleanly in one sentence, or when supplying an intentionally prepared +multi-part spec. Never generate one pre-emptively. + +If a spec file is genuinely needed, begin it with a markdown `# 3-5 word title` +line — a short overview of the task. The TUI agent viewer renders this as the job label; without it the viewer falls back to the first 5 words of the spec, which is rarely meaningful. diff --git a/packages/open-agents/README.md b/packages/open-agents/README.md index 6cda842..6a09946 100644 --- a/packages/open-agents/README.md +++ b/packages/open-agents/README.md @@ -8,7 +8,7 @@ persisted and re-printable. Not an orchestrator — the design goal is to **make a Cursor job a self-waiting, harness-trackable command** so the harness's own -background-completion notification *is* the orchestration: no polling, no +background-completion notification _is_ the orchestration: no polling, no status truncation, no near-miss re-send. ## Install @@ -31,37 +31,26 @@ different profile bound (read-only vs `--force` edit). ## Verbs ``` -submit --file a.md [--file b.md …] | "" [--edit] [--model m] [--timeout s] [--cwd d] +submit --task "" | --spec-file a.md [--spec-file b.md …] | "" + [--no-history] [--edit] [--model m] [--timeout s] [--cwd d] --model overrides the configured/provider default for this submission. default: BLOCKS until all done, exits worst code. Each job's clean deliverable → its OWN .result.md; stdout is just an index (`result → `) — no input echo, jobs never mix. Read the file(s). - many --file ⇒ run in parallel · --detach: print bare id(s), don't wait + --task and --spec-file may repeat; many inputs run in parallel + --file remains an alias for --spec-file · --detach: print bare id(s), don't wait --parallel-mode asap|all-together (default asap): asap prints each index line as its job finishes; all-together waits, submission order - --compact-mode: treat the inline input as a very short task title - (roughly 5-10 words) and let a Cursor spec-writer using the built-in - default model distill - recent conversation history into the executor spec. Do not paste - details, quotes, or file lists into the inline prompt when the current - conversation already has that context. - History resolves from --history-file , OPEN_AGENTS_HISTORY_FILE, - CLAUDE_CODE_TRANSCRIPT_PATH, or the current Claude Code transcript - when CLAUDE_CODE_SESSION_ID is set. It also detects Codex sessions - from CODEX_THREAD_ID / ~/.codex/sessions and falls back to - ~/.codex/history.jsonl when no richer transcript is available. - Compact mode gives the spec-writer the last 50000 token-like history - units plus a pointer to the full history file, or 100000 when a local - Headroom proxy is healthy. The generated spec is kept concise and points - back to the history file for uncertain detail instead of copying the - transcript. Progress output includes the spec generation duration and - a ~/... path to the generated spec artifact. - --headroom auto|off|require (default auto): probe the local Headroom - proxy and, when healthy, run the Cursor adapter with - OPENAI_BASE_URL=http://127.0.0.1:8787/v1. If the proxy is absent, - jobs continue normally with a concise stderr notice. Use - --headroom-url , OPEN_AGENTS_HEADROOM_URL, or config - headroom.url for a non-default proxy. + current-session history is automatic. Up to 100000 token-like units go + directly to each executor without a preparatory model run. Above that + budget, one foreground curator run returns an isolated relevant-history + slice for each task; the submitted task itself remains verbatim. + Exact sessions resolve from --history-file, explicit transcript env + variables, CLAUDE_CODE_SESSION_ID, or CODEX_THREAD_ID. The CLI never + guesses the newest transcript. If no exact session is available it + warns and continues task-only. --no-history disables the behavior. + --include-relevant-history and --compact-mode remain compatibility + aliases for the default behavior. wait wait for ALL running jobs (this repo) wait "" | wait for ALL of these wait --any return when ≥1 done; prints which (loop = popcorn) @@ -112,26 +101,6 @@ The bin launcher prefers `bun` (matches sibling davstack packages); set `OPEN_AGENTS_RUNTIME=node` to use `node --experimental-transform-types` instead. The source is pure `node:*` — either runtime works. -## Optional Headroom proxy - -Headroom is optional. By default, `open-agents` briefly probes -`http://127.0.0.1:8787/health`. If the proxy is healthy and the selected adapter -is `cursor`, the spawned Cursor Agent process receives -`OPENAI_BASE_URL=http://127.0.0.1:8787/v1`. If the proxy is not installed or not -running, the job still runs directly. - -Controls: - -```bash -OPEN_AGENTS_HEADROOM=off explore submit "short task" -OPEN_AGENTS_HEADROOM=require fast-edit submit --compact-mode "fix parser" -OPEN_AGENTS_HEADROOM_URL=http://127.0.0.1:8788 explore submit "inspect auth" -``` - -In compact mode, a healthy Headroom proxy raises the history tail budget from -50000 to 100000 token-like units and prints a `/stats` delta from -`requests.total` and `tokens.saved`. - ## Job state Lives under `~/.davstack/jobs//` (override `OPEN_AGENTS_HOME`). diff --git a/packages/open-agents/__tests__/compact-background.test.ts b/packages/open-agents/__tests__/compact-background.test.ts new file mode 100644 index 0000000..2b09766 --- /dev/null +++ b/packages/open-agents/__tests__/compact-background.test.ts @@ -0,0 +1,110 @@ +import { spawnSync } from "node:child_process"; +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { describe, expect, test } from "vitest"; +import type { JobRecord } from "../src/core/jobs.js"; +import { repoHash } from "../src/core/paths.js"; + +const exploreEntrypoint = fileURLToPath( + new URL("../src/entrypoints/explore.ts", import.meta.url), +); + +const delay = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms)); + +describe("automatic history background submission", () => { + test("accepts punctuation safely and returns one executor with direct history", async () => { + const sandbox = mkdtempSync( + join(tmpdir(), "open-agents-compact-background-"), + ); + const repo = join(sandbox, "repo"); + const state = join(sandbox, "state"); + const history = join(sandbox, "history.jsonl"); + mkdirSync(repo, { recursive: true }); + writeFileSync(join(repo, "package.json"), "{}\n"); + writeFileSync( + history, + `${JSON.stringify({ role: "user", content: "context" })}\n`, + ); + + try { + const task = `review Juno's "voice" via $path; keep | and & literal`; + const run = spawnSync( + process.execPath, + [ + "--import", + "tsx", + exploreEntrypoint, + "submit", + "--task", + task, + "--history-file", + history, + "--background", + "--cwd", + repo, + ], + { + cwd: dirname(dirname(dirname(exploreEntrypoint))), + encoding: "utf8", + env: { + ...process.env, + OPEN_AGENTS_HOME: state, + CURSOR_AGENT_BIN: join(sandbox, "missing-agent"), + }, + timeout: 10_000, + windowsHide: true, + }, + ); + + expect(run.error).toBeUndefined(); + expect(run.status).toBe(0); + const ids = run.stdout.trim().split(/\s+/).filter(Boolean); + expect(ids).toHaveLength(1); + + const jobDir = join(state, "jobs", repoHash(repo)); + const jobPath = join(jobDir, `${ids[0]}.json`); + expect(existsSync(jobPath)).toBe(true); + const accepted = JSON.parse(readFileSync(jobPath, "utf8")) as JobRecord; + expect(accepted.prompt).toBe(task); + expect(accepted.compactTask).toBeUndefined(); + expect(accepted.historyMode).toBe("direct"); + expect(accepted.fullPrompt).toContain("Read the complete delegated task"); + expect(["running", "failed"]).toContain(accepted.status); + + const spec = readFileSync(join(jobDir, `${ids[0]}.spec.md`), "utf8"); + expect(spec).toContain(`# Authoritative task\n${task}`); + expect(spec).toContain("User:\ncontext"); + + let settled = accepted; + for ( + let attempt = 0; + attempt < 50 && settled.status === "running"; + attempt += 1 + ) { + await delay(100); + settled = JSON.parse(readFileSync(jobPath, "utf8")) as JobRecord; + } + expect(settled.status).toBe("failed"); + + const records = readdirSync(jobDir) + .filter((name) => name.endsWith(".json")) + .map( + (name) => + JSON.parse(readFileSync(join(jobDir, name), "utf8")) as JobRecord, + ); + expect(records.filter((job) => job.prompt === task)).toHaveLength(1); + } finally { + rmSync(sandbox, { recursive: true, force: true }); + } + }, 20_000); +}); diff --git a/packages/open-agents/__tests__/compact-messages.test.ts b/packages/open-agents/__tests__/compact-messages.test.ts deleted file mode 100644 index 3c132f5..0000000 --- a/packages/open-agents/__tests__/compact-messages.test.ts +++ /dev/null @@ -1,92 +0,0 @@ -import { describe, expect, test } from 'vitest' -import { - compactMessageContent, - compactMessages, - renderCompactMessages, -} from '../src/core/compact.js' -import type { CompactMessage } from '../src/core/compact.js' - -function bigJsonToolResult(): string { - return JSON.stringify( - Array.from({ length: 80 }, (_, i) => ({ - id: i, - name: `row-${i}`, - value: i * 7, - active: i % 2 === 0, - note: 'lorem ipsum dolor sit amet consectetur adipiscing elit', - })), - ) -} - -describe('compactMessageContent', () => { - test('reduces a large JSON string content, preserves role and schema keys', () => { - const original = bigJsonToolResult() - const message: CompactMessage = { role: 'tool', content: original } - const result = compactMessageContent(message) - - expect(result.role).toBe('tool') - expect(typeof result.content).toBe('string') - expect((result.content as string).length).toBeLessThan(original.length) - // schema keys still present in the compacted output - expect(result.content as string).toContain('id') - expect(result.content as string).toContain('name') - expect(result.content as string).toContain('value') - }) - - test('passes a small / plain message through unchanged', () => { - const message: CompactMessage = { role: 'user', content: 'fix the enum bug please' } - const result = compactMessageContent(message) - expect(result.role).toBe('user') - expect(result.content).toBe('fix the enum bug please') - }) - - test('array content: text blocks compacted, non-text blocks preserved', () => { - const big = bigJsonToolResult() - const message: CompactMessage = { - role: 'assistant', - content: [ - { type: 'text', text: big }, - { type: 'tool_use', id: 'tu_1', name: 'do_thing', input: { a: 1 } }, - { type: 'text', content: big }, - ], - } - const result = compactMessageContent(message) - expect(result.role).toBe('assistant') - const blocks = result.content as Array> - - // text block compacted - expect((blocks[0].text as string).length).toBeLessThan(big.length) - expect(blocks[0].type).toBe('text') - // non-text block untouched - expect(blocks[1]).toEqual({ type: 'tool_use', id: 'tu_1', name: 'do_thing', input: { a: 1 } }) - // .content string block compacted too - expect((blocks[2].content as string).length).toBeLessThan(big.length) - }) -}) - -describe('compactMessages', () => { - test('aggregates token estimates and reduces a big tool_result message', () => { - const messages: CompactMessage[] = [ - { role: 'user', content: 'short ask' }, - { role: 'tool', content: bigJsonToolResult() }, - ] - const result = compactMessages(messages) - expect(result.messages).toHaveLength(2) - expect(result.tokensAfter).toBeLessThan(result.tokensBefore) - expect(result.tokensSaved).toBe(result.tokensBefore - result.tokensAfter) - expect(result.messages[0].role).toBe('user') - expect(result.messages[1].role).toBe('tool') - }) - - test('renderCompactMessages still works on compacted messages', () => { - const messages: CompactMessage[] = [ - { role: 'user', content: 'do the thing' }, - { role: 'tool', content: bigJsonToolResult() }, - ] - const { messages: compacted } = compactMessages(messages) - const rendered = renderCompactMessages(compacted) - expect(rendered).toContain('user: do the thing') - expect(rendered).toContain('tool:') - expect(rendered.length).toBeGreaterThan(0) - }) -}) diff --git a/packages/open-agents/__tests__/compact.test.ts b/packages/open-agents/__tests__/compact.test.ts deleted file mode 100644 index a0a7f54..0000000 --- a/packages/open-agents/__tests__/compact.test.ts +++ /dev/null @@ -1,147 +0,0 @@ -import { describe, expect, test } from 'vitest'; -import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; -import { tmpdir } from 'node:os'; -import { join } from 'node:path'; -import { - findClaudeTranscriptBySession, - findCodexTranscriptBySession, - loadCompactHistory, - parseCompactMessages, - renderCompactMessages, - resolveCompactHistoryFile, - tokenTail, -} from '../src/core/compact.js'; -import { parseFlags } from '../src/cli.js'; - -describe('compact mode helpers', () => { - test('tokenTail keeps the last token-like units', () => { - expect(tokenTail('one two three four', 2)).toBe('three four'); - expect(tokenTail('one, two. three!', 4)).toBe('two. three!'); - }); - - test('compact flags parse', () => { - const { flags, positional } = parseFlags([ - '--compact-mode', - '--history-file', - 'C:/history.jsonl', - 'short task', - ]); - expect(flags.compactMode).toBe(true); - expect(flags.historyFile).toBe('C:/history.jsonl'); - expect(positional).toEqual(['short task']); - }); - - test('model override parses explicitly and unsupported options stay out of the prompt', () => { - const parsed = parseFlags(['--model', 'provider-model', '--typo', 'short task']); - expect(parsed.flags.model).toBe('provider-model'); - expect(parsed.flags.unknownOptions).toEqual(['--typo']); - expect(parsed.positional).toEqual(['short task']); - }); - - test('loads real JSONL messages for compaction', () => { - const dir = mkdtempSync(join(tmpdir(), 'compact-messages-')); - try { - const history = join(dir, 'history.jsonl'); - writeFileSync( - history, - [ - JSON.stringify({ message: { role: 'user', content: 'keep user intent' } }), - JSON.stringify({ message: { role: 'assistant', content: 'assistant response' } }), - ].join('\n') + '\n', - ); - - const loaded = loadCompactHistory(history, 100); - - expect(loaded.messages).toEqual([ - { role: 'user', content: 'keep user intent' }, - { role: 'assistant', content: 'assistant response' }, - ]); - expect(loaded.tail).toContain('user: keep user intent'); - } finally { - rmSync(dir, { recursive: true, force: true }); - } - }); - - test('renders parsed compact messages without changing roles', () => { - const messages = parseCompactMessages( - [ - JSON.stringify({ role: 'user', content: 'do not compress this as tool output' }), - JSON.stringify({ role: 'assistant', content: [{ type: 'text', text: 'ok' }] }), - ].join('\n'), - ); - - expect(renderCompactMessages(messages)).toBe( - 'user: do not compress this as tool output\n\nassistant: ok', - ); - }); - - test('resolves explicit and env history paths before Claude session lookup', () => { - expect( - resolveCompactHistoryFile({ - historyFile: 'explicit.jsonl', - env: { OPEN_AGENTS_HISTORY_FILE: 'env.jsonl' }, - }), - ).toContain('explicit.jsonl'); - expect( - resolveCompactHistoryFile({ - env: { OPEN_AGENTS_HISTORY_FILE: 'env.jsonl' }, - }), - ).toContain('env.jsonl'); - }); - - test('finds Claude Code transcript by session id', () => { - const home = mkdtempSync(join(tmpdir(), 'compact-history-')); - try { - const sessionId = 'fe080f49-6b6b-4f57-98ab-c954671c6a40'; - const projectDir = join(home, '.claude', 'projects', 'C--Users-dpwra-dev-davstack'); - mkdirSync(projectDir, { recursive: true }); - const transcript = join(projectDir, `${sessionId}.jsonl`); - writeFileSync(transcript, '{}\n'); - - expect(findClaudeTranscriptBySession(sessionId, home)).toBe(transcript); - expect( - resolveCompactHistoryFile({ - env: { CLAUDE_CODE_SESSION_ID: sessionId }, - homeDir: home, - }), - ).toBe(transcript); - } finally { - rmSync(home, { recursive: true, force: true }); - } - }); - - test('finds Codex transcript by thread id', () => { - const home = mkdtempSync(join(tmpdir(), 'compact-codex-history-')); - try { - const threadId = '019eabb5-b1ba-7ba1-b84b-97efcca16393'; - const sessionDir = join(home, '.codex', 'sessions', '2026', '06', '09'); - mkdirSync(sessionDir, { recursive: true }); - const transcript = join(sessionDir, `rollout-2026-06-09T10-27-52-${threadId}.jsonl`); - writeFileSync(transcript, '{}\n'); - - expect(findCodexTranscriptBySession(threadId, home)).toBe(transcript); - expect( - resolveCompactHistoryFile({ - env: { CODEX_THREAD_ID: threadId }, - homeDir: home, - }), - ).toBe(transcript); - } finally { - rmSync(home, { recursive: true, force: true }); - } - }); - - test('falls back to Codex history.jsonl when no active session env is present', () => { - const home = mkdtempSync(join(tmpdir(), 'compact-codex-history-')); - try { - const codexDir = join(home, '.codex'); - mkdirSync(codexDir, { recursive: true }); - const history = join(codexDir, 'history.jsonl'); - writeFileSync(history, '{"text":"hello"}\n'); - - expect(resolveCompactHistoryFile({ env: {}, homeDir: home })).toBe(history); - } finally { - rmSync(home, { recursive: true, force: true }); - } - }); -}); diff --git a/packages/open-agents/__tests__/history.test.ts b/packages/open-agents/__tests__/history.test.ts new file mode 100644 index 0000000..d0bff32 --- /dev/null +++ b/packages/open-agents/__tests__/history.test.ts @@ -0,0 +1,248 @@ +import { describe, expect, test } from "vitest"; +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + buildHistoryCuratorTask, + DEFAULT_DIRECT_HISTORY_TOKENS, + findClaudeTranscriptBySession, + findCodexTranscriptBySession, + historyNeedsCuration, + loadConversationHistory, + parseConversationMessages, + parseCuratedHistory, + resolveConversationHistoryFile, +} from "../src/core/history.js"; +import { parseFlags } from "../src/cli.js"; + +describe("conversation history helpers", () => { + test("compact flags parse", () => { + const { flags, positional } = parseFlags([ + "--compact-mode", + "--history-file", + "C:/history.jsonl", + "short task", + ]); + expect(flags.compactMode).toBe(true); + expect(flags.historyFile).toBe("C:/history.jsonl"); + expect(positional).toEqual(["short task"]); + }); + + test("automatic history flags and task aliases parse", () => { + const parsed = parseFlags([ + "--task", + "review architecture", + "--spec-file", + "follow-up.md", + "--include-relevant-history", + ]); + expect(parsed.flags.tasks).toEqual(["review architecture"]); + expect(parsed.flags.files).toEqual(["follow-up.md"]); + expect(parsed.flags.includeRelevantHistory).toBe(true); + expect(parseFlags(["--no-history", "task"]).flags.noHistory).toBe(true); + }); + + test("model override parses explicitly and unsupported options stay out of the prompt", () => { + const parsed = parseFlags([ + "--model", + "provider-model", + "--typo", + "short task", + ]); + expect(parsed.flags.model).toBe("provider-model"); + expect(parsed.flags.unknownOptions).toEqual(["--typo"]); + expect(parsed.positional).toEqual(["short task"]); + }); + + test("extracts visible Codex conversation events without tool trace noise", () => { + const messages = parseConversationMessages( + [ + JSON.stringify({ + type: "response_item", + payload: { + type: "custom_tool_call_output", + output: "large tool output", + }, + }), + JSON.stringify({ + type: "event_msg", + payload: { type: "user_message", message: "review the voice design" }, + }), + JSON.stringify({ + type: "event_msg", + payload: { + type: "agent_message", + message: "I will inspect it", + phase: "commentary", + }, + }), + ].join("\n"), + ); + + expect(messages).toEqual([ + { role: "user", content: "review the voice design" }, + { role: "assistant", content: "I will inspect it" }, + ]); + }); + + test("extracts visible Claude messages and omits tool blocks", () => { + const messages = parseConversationMessages( + [ + JSON.stringify({ + message: { + role: "user", + content: [ + { type: "text", text: "keep this request" }, + { type: "tool_result", content: "omit tool output" }, + ], + }, + }), + JSON.stringify({ + message: { role: "assistant", content: "keep this response" }, + }), + ].join("\n"), + ); + + expect(messages).toEqual([ + { role: "user", content: "keep this request" }, + { role: "assistant", content: "keep this response" }, + ]); + }); + + test("loads the full visible conversation without compacting or raw-JSON fallback", () => { + const dir = mkdtempSync(join(tmpdir(), "conversation-history-")); + try { + const historyFile = join(dir, "rollout.jsonl"); + writeFileSync( + historyFile, + [ + JSON.stringify({ + type: "event_msg", + payload: { + type: "user_message", + message: "keep this exact request", + }, + }), + JSON.stringify({ + type: "event_msg", + payload: { type: "agent_message", message: "and this response" }, + }), + ].join("\n"), + ); + const history = loadConversationHistory(historyFile); + expect(history.text).toContain("User:\nkeep this exact request"); + expect(history.text).toContain("Assistant:\nand this response"); + expect(history.tokens).toBeGreaterThan(0); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("curates only above the direct-history budget", () => { + expect(historyNeedsCuration(DEFAULT_DIRECT_HISTORY_TOKENS)).toBe(false); + expect(historyNeedsCuration(DEFAULT_DIRECT_HISTORY_TOKENS + 1)).toBe(true); + }); + + test("keeps curator output isolated by task id", () => { + const task = buildHistoryCuratorTask({ + tasks: [ + { taskId: "task-1", task: "review voice" }, + { taskId: "task-2", task: "rename adapter" }, + ], + historyPath: "C:/history.jsonl", + historyText: "User:\ncontext", + }); + expect(task).toContain("Do not create\none shared summary"); + const contexts = parseCuratedHistory( + JSON.stringify({ + contexts: [ + { taskId: "task-1", relevantHistory: "voice only" }, + { taskId: "task-2", relevantHistory: "adapter only" }, + ], + }), + ["task-1", "task-2"], + ); + expect(contexts.get("task-1")).toBe("voice only"); + expect(contexts.get("task-2")).toBe("adapter only"); + }); + + test("resolves explicit and env history paths before Claude session lookup", () => { + expect( + resolveConversationHistoryFile({ + historyFile: "explicit.jsonl", + env: { OPEN_AGENTS_HISTORY_FILE: "env.jsonl" }, + }), + ).toContain("explicit.jsonl"); + expect( + resolveConversationHistoryFile({ + env: { OPEN_AGENTS_HISTORY_FILE: "env.jsonl" }, + }), + ).toContain("env.jsonl"); + }); + + test("finds Claude Code transcript by session id", () => { + const home = mkdtempSync(join(tmpdir(), "compact-history-")); + try { + const sessionId = "fe080f49-6b6b-4f57-98ab-c954671c6a40"; + const projectDir = join( + home, + ".claude", + "projects", + "C--Users-dpwra-dev-davstack", + ); + mkdirSync(projectDir, { recursive: true }); + const transcript = join(projectDir, `${sessionId}.jsonl`); + writeFileSync(transcript, "{}\n"); + + expect(findClaudeTranscriptBySession(sessionId, home)).toBe(transcript); + expect( + resolveConversationHistoryFile({ + env: { CLAUDE_CODE_SESSION_ID: sessionId }, + homeDir: home, + }), + ).toBe(transcript); + } finally { + rmSync(home, { recursive: true, force: true }); + } + }); + + test("finds Codex transcript by thread id", () => { + const home = mkdtempSync(join(tmpdir(), "compact-codex-history-")); + try { + const threadId = "019eabb5-b1ba-7ba1-b84b-97efcca16393"; + const sessionDir = join(home, ".codex", "sessions", "2026", "06", "09"); + mkdirSync(sessionDir, { recursive: true }); + const transcript = join( + sessionDir, + `rollout-2026-06-09T10-27-52-${threadId}.jsonl`, + ); + writeFileSync(transcript, "{}\n"); + + expect(findCodexTranscriptBySession(threadId, home)).toBe(transcript); + expect( + resolveConversationHistoryFile({ + env: { CODEX_THREAD_ID: threadId }, + homeDir: home, + }), + ).toBe(transcript); + } finally { + rmSync(home, { recursive: true, force: true }); + } + }); + + test("does not guess a newest Codex history when no active session id is present", () => { + const home = mkdtempSync(join(tmpdir(), "compact-codex-history-")); + try { + const codexDir = join(home, ".codex"); + mkdirSync(codexDir, { recursive: true }); + const history = join(codexDir, "history.jsonl"); + writeFileSync(history, '{"text":"hello"}\n'); + + expect( + resolveConversationHistoryFile({ env: {}, homeDir: home }), + ).toBeNull(); + } finally { + rmSync(home, { recursive: true, force: true }); + } + }); +}); diff --git a/packages/open-agents/package.json b/packages/open-agents/package.json index 87cbd9b..ddedd85 100644 --- a/packages/open-agents/package.json +++ b/packages/open-agents/package.json @@ -58,8 +58,7 @@ "bun": ">=1.1" }, "dependencies": { - "@davstack/cli-utils": "workspace:*", - "@davstack/context-compactor": "workspace:*" + "@davstack/cli-utils": "workspace:*" }, "devDependencies": { "@types/node": "^25.9.1", diff --git a/packages/open-agents/src/cli.ts b/packages/open-agents/src/cli.ts index df1c846..e8778f8 100644 --- a/packages/open-agents/src/cli.ts +++ b/packages/open-agents/src/cli.ts @@ -11,39 +11,45 @@ // scaffolds live in profiles/. This file only parses flags, picks an adapter // (default cursor) + profile, and dispatches verbs through core/. -import { spawn } from 'node:child_process'; -import { existsSync, readFileSync, writeFileSync } from 'node:fs'; -import { homedir } from 'node:os'; -import { join } from 'node:path'; -import { fileURLToPath } from 'node:url'; -import { agyAdapter } from './adapters/agy.js'; -import { cursorAdapter } from './adapters/cursor.js'; -import { geminiAdapter } from './adapters/gemini.js'; -import type { AgentAdapter } from './adapters/types.js'; +import { spawn } from "node:child_process"; +import { existsSync, readFileSync, writeFileSync } from "node:fs"; +import { homedir } from "node:os"; +import { join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { agyAdapter } from "./adapters/agy.js"; +import { cursorAdapter } from "./adapters/cursor.js"; +import { geminiAdapter } from "./adapters/gemini.js"; +import type { AgentAdapter } from "./adapters/types.js"; import { - buildCompactSpecWriterTask, - loadCompactHistory, - resolveCompactHistoryFile, -} from './core/compact.js'; -import { readDeliverable, renderJobResult } from './core/deliverable.js'; + buildHistoryCuratorTask, + DEFAULT_DIRECT_HISTORY_TOKENS, + historyNeedsCuration, + loadConversationHistory, + parseCuratedHistory, + resolveConversationHistoryFile, + type ConversationHistory, +} from "./core/history.js"; +import { readDeliverable, renderJobResult } from "./core/deliverable.js"; import { createJob, listJobs, mostRecentFinishedJob, readJob, updateJob, -} from './core/jobs.js'; -import { jobsDir } from './core/paths.js'; -import { DEFAULT_TIMEOUT_SEC, runJob } from './core/run.js'; -import { editProfile } from './profiles/edit.js'; -import { exploreProfile } from './profiles/explore.js'; -import { specWriterProfile } from './profiles/spec-writer.js'; -import type { Profile } from './profiles/types.js'; -import { loadConfig } from './config.js'; -import { runCheck } from './check.js'; - -const SELF = fileURLToPath(import.meta.url); -const TERMINAL = new Set(['done', 'failed', 'cancelled']); +} from "./core/jobs.js"; +import { jobsDir } from "./core/paths.js"; +import { DEFAULT_TIMEOUT_SEC, runJob } from "./core/run.js"; +import { editProfile } from "./profiles/edit.js"; +import { exploreProfile } from "./profiles/explore.js"; +import { historyCuratorProfile } from "./profiles/history-curator.js"; +import type { Profile } from "./profiles/types.js"; +import { loadConfig } from "./config.js"; +import { runCheck } from "./check.js"; + +const SELF = process.argv[1] + ? resolve(process.argv[1]) + : fileURLToPath(import.meta.url); +const TERMINAL = new Set(["done", "failed", "cancelled"]); const ADAPTERS: Record = { cursor: cursorAdapter, @@ -53,7 +59,7 @@ const ADAPTERS: Record = { function genId(): string { const d = new Date(); - const p = (n: number) => String(n).padStart(2, '0'); + const p = (n: number) => String(n).padStart(2, "0"); const stamp = `${d.getFullYear()}${p(d.getMonth() + 1)}${p(d.getDate())}` + `-${p(d.getHours())}${p(d.getMinutes())}${p(d.getSeconds())}`; @@ -68,6 +74,7 @@ export interface Flags { detach?: boolean; noInline?: boolean; files?: string[]; + tasks?: string[]; model?: string; cwd?: string; timeout?: number; @@ -75,36 +82,47 @@ export interface Flags { adapter?: string; compactMode?: boolean; historyFile?: string; + noHistory?: boolean; + includeRelevantHistory?: boolean; json?: boolean; unknownOptions?: string[]; } -export function parseFlags(argv: string[]): { flags: Flags; positional: string[] } { +export function parseFlags(argv: string[]): { + flags: Flags; + positional: string[]; +} { const flags: Flags = {}; const positional: string[] = []; for (let i = 0; i < argv.length; i += 1) { let a = argv[i]; let inlineVal: string | undefined; - if (a.startsWith('--') && a.includes('=')) { - const j = a.indexOf('='); + if (a.startsWith("--") && a.includes("=")) { + const j = a.indexOf("="); inlineVal = a.slice(j + 1); a = a.slice(0, j); } const val = () => (inlineVal !== undefined ? inlineVal : argv[++i]); - if (a === '--edit' || a === '--any' || a === '--all' || a === '--detach') + if (a === "--edit" || a === "--any" || a === "--all" || a === "--detach") (flags as any)[a.slice(2)] = true; - else if (a === '--compact-mode') flags.compactMode = true; - else if (a === '--background' || a === '--bg' || a === '--no-wait') flags.detach = true; - else if (a === '--no-inline') flags.noInline = true; - else if (a === '--file') (flags.files ||= []).push(val()); - else if (a === '--model') flags.model = val(); - else if (a === '--cwd') flags.cwd = val(); - else if (a === '--timeout') flags.timeout = Number(val()); - else if (a === '--parallel-mode') flags.parallelMode = val(); - else if (a === '--adapter' || a === '--provider') flags.adapter = val(); - else if (a === '--history-file') flags.historyFile = val(); - else if (a === '--json') flags.json = true; - else if (a.startsWith('--')) (flags.unknownOptions ||= []).push(a); + else if (a === "--compact-mode") flags.compactMode = true; + else if (a === "--include-relevant-history") + flags.includeRelevantHistory = true; + else if (a === "--no-history") flags.noHistory = true; + else if (a === "--background" || a === "--bg" || a === "--no-wait") + flags.detach = true; + else if (a === "--no-inline") flags.noInline = true; + else if (a === "--file" || a === "--spec-file") + (flags.files ||= []).push(val()); + else if (a === "--task") (flags.tasks ||= []).push(val()); + else if (a === "--model") flags.model = val(); + else if (a === "--cwd") flags.cwd = val(); + else if (a === "--timeout") flags.timeout = Number(val()); + else if (a === "--parallel-mode") flags.parallelMode = val(); + else if (a === "--adapter" || a === "--provider") flags.adapter = val(); + else if (a === "--history-file") flags.historyFile = val(); + else if (a === "--json") flags.json = true; + else if (a.startsWith("--")) (flags.unknownOptions ||= []).push(a); else positional.push(argv[i]); } return { flags, positional }; @@ -118,17 +136,25 @@ export function parseFlags(argv: string[]): { flags: Flags; positional: string[] // normalized to a trailing newline so the scaffold's `` separator stays // on its own line; an adapter addendum is already expected to be newline- // terminated by its author. -export function combineAddendums(adapterAddendum: string, userExtension: string): string { +export function combineAddendums( + adapterAddendum: string, + userExtension: string, +): string { const normalized = - userExtension && !userExtension.endsWith('\n') ? userExtension + '\n' : userExtension; + userExtension && !userExtension.endsWith("\n") + ? userExtension + "\n" + : userExtension; return adapterAddendum + normalized; } -export function pickAdapter(flags: Flags, configAdapter?: string): AgentAdapter { +export function pickAdapter( + flags: Flags, + configAdapter?: string, +): AgentAdapter { // Default: cursor. Unknown adapter names fall back to cursor // too (no silent gemini on a typo). Flag wins over config; config wins over // built-in default. - const name = flags.adapter || configAdapter || 'cursor'; + const name = flags.adapter || configAdapter || "cursor"; return ADAPTERS[name] || cursorAdapter; } @@ -144,105 +170,148 @@ function pickProfile(flags: Flags): Profile { } // --- submit ---------------------------------------------------------------- -const SHELL_HOSTILE = /[\n"'`$();|&<>]/; -const DEFAULT_COMPACT_TAIL_TOKENS = 50000; - function userPath(path: string): string { const home = homedir(); - const normalized = path.replace(/\\/g, '/'); - const normalizedHome = home.replace(/\\/g, '/'); - if (normalized === normalizedHome) return '~'; + const normalized = path.replace(/\\/g, "/"); + const normalizedHome = home.replace(/\\/g, "/"); + if (normalized === normalizedHome) return "~"; if (normalized.startsWith(`${normalizedHome}/`)) { return `~/${normalized.slice(normalizedHome.length + 1)}`; } return normalized; } -function compactHistoryFile(flags: Flags): string { - const historyFile = resolveCompactHistoryFile({ historyFile: flags.historyFile }); +function resolveConversationHistory(flags: Flags): ConversationHistory | null { + if (flags.noHistory) return null; + const historyFile = resolveConversationHistoryFile({ + historyFile: flags.historyFile, + }); if (!historyFile) { - throw new Error( - 'open-agents submit --compact-mode needs history context. Pass --history-file , set OPEN_AGENTS_HISTORY_FILE or CLAUDE_CODE_TRANSCRIPT_PATH, or run from a Claude Code session with CLAUDE_CODE_SESSION_ID.', + process.stderr.write( + "open-agents: no exact current-session history found; continuing task-only (use --history-file to provide one)\n", + ); + return null; + } + const history = loadConversationHistory(historyFile); + if (!history.text) { + process.stderr.write( + `open-agents: current-session history contained no visible user/assistant messages; continuing task-only (${userPath(history.path)})\n`, ); + return null; } - return historyFile; + return history; } -async function generateCompactSpec(input: { +async function generateRelevantHistory(input: { repoPath: string; timeoutSec: number; - targetProfile: Profile; - task: string; - historyFile: string; - tailTokens: number; -}): Promise { - const history = loadCompactHistory(input.historyFile, input.tailTokens); + tasks: Array<{ taskId: string; task: string }>; + history: ConversationHistory; +}): Promise> { const adapter = cursorAdapter; const model = cursorAdapter.defaultModel(); - const historyTail = history.tail; - const specWriterTask = buildCompactSpecWriterTask({ - task: input.task, - repoPath: input.repoPath, - targetProfile: input.targetProfile.name, - historyPath: history.path, - historyTail, - tailTokens: input.tailTokens, + const curatorTask = buildHistoryCuratorTask({ + tasks: input.tasks, + historyPath: input.history.path, + historyText: input.history.text, }); const id = genId(); - const specWriterTaskPath = join(jobsDir(input.repoPath), `${id}.spec.md`); + const curatorTaskPath = join(jobsDir(input.repoPath), `${id}.history.md`); createJob({ id, repoPath: input.repoPath, - prompt: `compact spec: ${input.task}`.slice(0, 500), + prompt: `select relevant history for ${input.tasks.length} task(s)`, model, background: true, }); try { - writeFileSync(specWriterTaskPath, specWriterTask.trim() + '\n', 'utf8'); + writeFileSync(curatorTaskPath, curatorTask.trim() + "\n", "utf8"); } catch { /* best-effort */ } - const specWriterWrapper = [ - 'Read the compact-mode spec-writer task from this file:', - '', - specWriterTaskPath, - '', - 'The file contains the short task, repository path, history pointer, and recent history tail.', - 'Use it as the authoritative task source and produce only the generated executor spec.', - ].join('\n'); + const curatorWrapper = [ + "Read the complete history-selection task from this file:", + "", + curatorTaskPath, + "", + "Return only the strict JSON mapping requested by that file.", + ].join("\n"); updateJob(input.repoPath, id, { - fullPrompt: specWriterProfile.buildPrompt(specWriterWrapper), + fullPrompt: historyCuratorProfile.buildPrompt(curatorWrapper), edit: false, model, timeoutSec: input.timeoutSec, + historyMode: "curated", + historyTokens: input.history.tokens, }); process.stderr.write( - `open-agents: compact-mode spec writer running (${adapter.name}, ${model}, ${history.tailTokens} history tokens) ...\n`, + `open-agents: history exceeds ${DEFAULT_DIRECT_HISTORY_TOKENS} tokens; selecting context for ${input.tasks.length} task(s) in one curator run (${adapter.name}, ${model}) ...\n`, ); const preToken = adapter.preSpawn(input.repoPath); - const specStartedAt = Date.now(); + const curatorStartedAt = Date.now(); await runJob( - { adapter, profile: specWriterProfile, env: {} }, + { adapter, profile: historyCuratorProfile, env: {} }, input.repoPath, id, ); - const specElapsedSec = Math.round((Date.now() - specStartedAt) / 1000); + const curatorElapsedSec = Math.round((Date.now() - curatorStartedAt) / 1000); adapter.postExit(input.repoPath, preToken); const job = readJob(input.repoPath, id); - if (!job || job.status !== 'done') { - throw new Error(`compact-mode spec writer failed: ${id}`); + if (!job || job.status !== "done") { + throw new Error(`history curator failed: ${id}`); } const generated = readDeliverable(adapter, job).trim(); - if (!generated || generated === '(no final message captured)') { - throw new Error(`compact-mode spec writer produced no spec: ${id}`); + if (!generated || generated === "(no final message captured)") { + throw new Error(`history curator produced no context: ${id}`); } + const contexts = parseCuratedHistory( + generated, + input.tasks.map(({ taskId }) => taskId), + ); process.stderr.write( - `open-agents: compact-mode generated spec ${id} in ${specElapsedSec}s (${generated.length} chars) -> ${userPath(job.resultPath || '')}\n`, + `open-agents: selected task-specific history in ${curatorElapsedSec}s -> ${userPath(job.resultPath || "")}\n`, ); - return generated; + return contexts; +} + +function buildExecutorSpec(task: string, relevantHistory: string): string { + if (!relevantHistory.trim()) return task.trim(); + return [ + "# Authoritative task", + task.trim(), + "", + "# Supporting conversation history", + "Use this only to interpret the task. Newer decisions override older ones.", + relevantHistory.trim(), + ].join("\n"); +} + +function buildExecutorFileWrapper(specPath: string): string { + return [ + "Read the complete delegated task and its optional conversation context from:", + "", + specPath, + "", + "The task section is authoritative. Use conversation history only as supporting context, then execute the task.", + ].join("\n"); +} + +function spawnDetachedRuns(ids: string[], repoPath: string): void { + for (const id of ids) { + spawn( + process.execPath, + [...process.execArgv, SELF, "__run", id, repoPath], + { + detached: true, + stdio: "ignore", + windowsHide: true, + env: { ...process.env }, + }, + ).unref(); + } } async function cmdSubmit(flags: Flags, positional: string[]): Promise { @@ -256,103 +325,151 @@ async function cmdSubmit(flags: Flags, positional: string[]): Promise { const model = flags.model || config.defaultModel || adapter.defaultModel(); const timeoutSec = Number.isFinite(flags.timeout) ? flags.timeout! - : (config.defaultTimeoutSec ?? DEFAULT_TIMEOUT_SEC); - const compactTailTokens = DEFAULT_COMPACT_TAIL_TOKENS; - const adapterAddendum = adapter.guardAddendum?.(profile.name, model) ?? ''; - const profileKey = profile.name as 'explore' | 'edit'; - const userExtension = config.profiles?.[profileKey]?.systemPromptExtension ?? ''; + : config.defaultTimeoutSec ?? DEFAULT_TIMEOUT_SEC; + const adapterAddendum = adapter.guardAddendum?.(profile.name, model) ?? ""; + const profileKey = profile.name as "explore" | "edit"; + const userExtension = + config.profiles?.[profileKey]?.systemPromptExtension ?? ""; const guardAddendum = combineAddendums(adapterAddendum, userExtension); - // adapter pre-run hook (cursor: clear stale .test.ts litter + snapshot, so - // the post hook removes only what THIS submit leaks). Detach's only cleanup. - const preToken = adapter.preSpawn(repoPath); - // Gather one or more spec bodies. Multiple --file → run them in parallel. - let bodies: string[] = []; + const bodies: string[] = []; const files = flags.files || []; - if (files.length) { - for (const f of files) { - if (!existsSync(f)) { - process.stderr.write(`open-agents: spec file not found: ${f}\n`); - process.exit(2); - } - bodies.push(readFileSync(f, 'utf8')); - } - } else { - const body = positional.join(' ').trim(); - if (!body) { - process.stderr.write('open-agents submit: need --file (or an inline prompt)\n'); - process.exit(2); - } - if (SHELL_HOSTILE.test(body)) { - process.stderr.write( - 'open-agents submit: inline prompt has shell-hostile chars. Write it to a ' + - 'file and use --file instead.\n', - ); + for (const f of files) { + if (!existsSync(f)) { + process.stderr.write(`open-agents: spec file not found: ${f}\n`); process.exit(2); } + bodies.push(readFileSync(f, "utf8")); + } + const inlineBodies = [...(flags.tasks || [])]; + const positionalBody = positional.join(" ").trim(); + if (positionalBody) inlineBodies.push(positionalBody); + for (const body of inlineBodies) { bodies.push(body); } + if (!bodies.length) { + process.stderr.write( + "open-agents submit: need --task , --spec-file , or an inline prompt\n", + ); + process.exit(2); + } - if (flags.compactMode) { - const historyFile = compactHistoryFile(flags); - bodies = await Promise.all( - bodies.map((body) => - generateCompactSpec({ - repoPath, - timeoutSec, - targetProfile: profile, - task: body, - historyFile, - tailTokens: compactTailTokens, - }), - ), + const history = resolveConversationHistory(flags); + const needsCuration = Boolean( + history && historyNeedsCuration(history.tokens), + ); + if (history && !needsCuration) { + process.stderr.write( + `open-agents: including current-session history directly (${history.tokens} token-like units, ${userPath(history.path)})\n`, ); } - const ids = bodies.map((body) => { - profile.warnIfMissingAcceptance(body); + const createRunRecord = ( + task: string, + relevantHistory: string, + historyMode: "none" | "direct" | "curated", + pendingHistory?: ConversationHistory, + ): string => { + profile.warnIfMissingAcceptance(task); const id = genId(); - createJob({ id, repoPath, prompt: body.trim().slice(0, 500), model, background: true }); + createJob({ + id, + repoPath, + prompt: task.trim().slice(0, 500), + model, + background: true, + }); + const specPath = join(jobsDir(repoPath), `${id}.spec.md`); + if (!pendingHistory) { + writeFileSync( + specPath, + buildExecutorSpec(task, relevantHistory) + "\n", + "utf8", + ); + } updateJob(repoPath, id, { - fullPrompt: profile.buildPrompt(body, guardAddendum), - edit: profile.mode === 'force', + ...(pendingHistory + ? { + historyTask: task, + historySourceFile: pendingHistory.path, + } + : { + fullPrompt: profile.buildPrompt( + buildExecutorFileWrapper(specPath), + guardAddendum, + ), + }), + edit: profile.mode === "force", model, timeoutSec, + adapterName: adapter.name, + historyMode, + historyTokens: pendingHistory?.tokens ?? history?.tokens, }); - // Durable prompt record: spec beside its .result.md, paired by job id. - try { - writeFileSync(join(jobsDir(repoPath), `${id}.spec.md`), body.trim() + '\n', 'utf8'); - } catch { - /* best-effort */ - } return id; + }; + + // Oversized detached submissions keep the durable-ID-before-work guarantee. + // Each runner selects its own context because batching would delay ID return + // and recreate the duplicate-submit failure mode. + if (flags.detach && history && needsCuration) { + const ids = bodies.map((task) => + createRunRecord(task, "", "curated", history), + ); + spawnDetachedRuns(ids, repoPath); + process.stdout.write(ids.join("\n") + "\n"); + return; + } + + const tasks = bodies.map((task, index) => ({ + taskId: `task-${index + 1}`, + task, + })); + let taskContexts = new Map(); + if (history && needsCuration) { + taskContexts = await generateRelevantHistory({ + repoPath, + timeoutSec, + tasks, + history, + }); + } + + const ids = tasks.map(({ taskId, task }) => { + const relevantHistory = history + ? needsCuration + ? taskContexts.get(taskId) ?? "" + : history.text + : ""; + return createRunRecord( + task, + relevantHistory, + history ? (needsCuration ? "curated" : "direct") : "none", + ); }); const deps = { adapter, profile, env: {} }; if (flags.detach) { - for (const id of ids) { - spawn(process.execPath, [SELF, '__run', id, repoPath], { - detached: true, - stdio: 'ignore', - windowsHide: true, - env: { ...process.env }, - }).unref(); - } - process.stdout.write(ids.join('\n') + '\n'); + spawnDetachedRuns(ids, repoPath); + process.stdout.write(ids.join("\n") + "\n"); return; } - const mode = (flags.parallelMode || 'asap').toLowerCase(); - if (mode !== 'asap' && mode !== 'all-together') { - process.stderr.write(`open-agents: --parallel-mode must be asap|all-together\n`); + const preToken = adapter.preSpawn(repoPath); + + const mode = (flags.parallelMode || "asap").toLowerCase(); + if (mode !== "asap" && mode !== "all-together") { + process.stderr.write( + `open-agents: --parallel-mode must be asap|all-together\n`, + ); process.exit(2); } const t0 = Date.now(); process.stderr.write( - `open-agents: ${ids.length} job(s) running (${profile.mode === 'force' ? 'edit' : 'explore'}, ${model}` + - `${ids.length > 1 ? `, ${mode}` : ''})…\n`, + `open-agents: ${ids.length} job(s) running (${profile.mode === "force" ? "edit" : "explore"}, ${model}` + + `${ids.length > 1 ? `, ${mode}` : ""})…\n`, ); let worst = 0; let printed = 0; @@ -372,10 +489,10 @@ async function cmdSubmit(flags: Flags, positional: string[]): Promise { } } } - process.stdout.write((printed++ ? '\n\n' : '') + body + '\n'); + process.stdout.write((printed++ ? "\n\n" : "") + body + "\n"); }; - if (mode === 'asap' && ids.length > 1) { + if (mode === "asap" && ids.length > 1) { let done = 0; await Promise.all( ids.map((id) => @@ -392,12 +509,14 @@ async function cmdSubmit(flags: Flags, positional: string[]): Promise { await Promise.all(ids.map((id) => runJob(deps, repoPath, id))); for (const id of ids) write(id); } - const paths = ids.map((id) => readJob(repoPath, id)?.resultPath).filter(Boolean); + const paths = ids + .map((id) => readJob(repoPath, id)?.resultPath) + .filter(Boolean); if (paths.length && !inline) { process.stdout.write( - '\n--- deliverable file(s) — read each for the actual output ---\n' + - paths.join('\n') + - '\n', + "\n--- deliverable file(s) — read each for the actual output ---\n" + + paths.join("\n") + + "\n", ); } adapter.postExit(repoPath, preToken); @@ -417,10 +536,80 @@ async function cmdRun(positional: string[], flags: Flags): Promise { const [id, repoPath] = positional; // A detached runner re-derives the profile from the persisted job record so // it does not need the entrypoint binding to have re-run. - const adapter = pickAdapter(flags); const job = readJob(repoPath, id); + if (!job) process.exit(1); + const adapter = + (job.adapterName && ADAPTERS[job.adapterName]) || pickAdapter(flags); const profile = FORCED_PROFILE || (job?.edit ? editProfile : exploreProfile); + + const pendingHistoryTask = job.historyTask ?? job.compactTask; + const pendingHistoryFile = job.historySourceFile ?? job.compactHistoryFile; + if (pendingHistoryTask) { + try { + if (!pendingHistoryFile) + throw new Error("history source path is missing"); + const history = loadConversationHistory(pendingHistoryFile); + const curated = historyNeedsCuration(history.tokens); + const relevantHistory = curated + ? ( + await generateRelevantHistory({ + repoPath, + timeoutSec: job.timeoutSec ?? DEFAULT_TIMEOUT_SEC, + tasks: [{ taskId: "task-1", task: pendingHistoryTask }], + history, + }) + ).get("task-1") ?? "" + : history.text; + const config = await loadConfig(repoPath); + const adapterAddendum = + adapter.guardAddendum?.(profile.name, job.model) ?? ""; + const profileKey = profile.name as "explore" | "edit"; + const userExtension = + config.profiles?.[profileKey]?.systemPromptExtension ?? ""; + const specPath = join(jobsDir(repoPath), `${id}.spec.md`); + writeFileSync( + specPath, + buildExecutorSpec(pendingHistoryTask, relevantHistory) + "\n", + "utf8", + ); + updateJob(repoPath, id, { + fullPrompt: profile.buildPrompt( + buildExecutorFileWrapper(specPath), + combineAddendums(adapterAddendum, userExtension), + ), + historyTask: undefined, + historySourceFile: undefined, + compactTask: undefined, + compactHistoryFile: undefined, + historyMode: curated ? "curated" : "direct", + historyTokens: history.tokens, + }); + } catch (error) { + const detail = error instanceof Error ? error.message : String(error); + const resultPath = join(jobsDir(repoPath), `${id}.result.md`); + try { + writeFileSync( + resultPath, + `history preparation failed: ${detail}\n`, + "utf8", + ); + } catch { + /* best-effort */ + } + updateJob(repoPath, id, { + status: "failed", + exitCode: 1, + finishedAt: new Date().toISOString(), + summary: `history preparation failed: ${detail}`, + resultPath, + }); + process.exit(1); + } + } + + const preToken = adapter.preSpawn(repoPath); await runJob({ adapter, profile }, repoPath, id); + adapter.postExit(repoPath, preToken); process.exit(0); } @@ -428,20 +617,24 @@ async function cmdRun(positional: string[], flags: Flags): Promise { async function cmdWait(flags: Flags, positional: string[]): Promise { const repoPath = flags.cwd || process.cwd(); const timeoutMs = - (Number.isFinite(flags.timeout) ? flags.timeout! : DEFAULT_TIMEOUT_SEC + 120) * 1000; + (Number.isFinite(flags.timeout) + ? flags.timeout! + : DEFAULT_TIMEOUT_SEC + 120) * 1000; const deadline = Date.now() + timeoutMs; const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); let ids = positional.flatMap((s) => String(s).split(/\s+/)).filter(Boolean); if (ids.length === 0) { ids = listJobs(repoPath) - .filter((j: any) => j.status === 'running') + .filter((j: any) => j.status === "running") .map((j: any) => j.id); if (ids.length === 0) process.exit(0); } else { const unknown = ids.filter((id) => !readJob(repoPath, id)); if (unknown.length) { - process.stderr.write(`open-agents wait: unknown job id(s): ${unknown.join(', ')}\n`); + process.stderr.write( + `open-agents wait: unknown job id(s): ${unknown.join(", ")}\n`, + ); process.exit(2); } } @@ -455,14 +648,14 @@ async function cmdWait(flags: Flags, positional: string[]): Promise { for (;;) { const finished = terminalIds(); if (flags.any && finished.length) { - process.stdout.write(finished.join('\n') + '\n'); + process.stdout.write(finished.join("\n") + "\n"); process.exit(0); } if (!flags.any && finished.length === ids.length) { process.exit(0); } if (Date.now() >= deadline) { - process.stderr.write('open-agents wait: timed out\n'); + process.stderr.write("open-agents wait: timed out\n"); process.exit(3); } await sleep(1500); @@ -470,29 +663,41 @@ async function cmdWait(flags: Flags, positional: string[]): Promise { } // --- result ---------------------------------------------------------------- -function printJobResult(adapter: AgentAdapter, repoPath: string, id: string | null): void { +function printJobResult( + adapter: AgentAdapter, + repoPath: string, + id: string | null, +): void { const job = id ? readJob(repoPath, id) : mostRecentFinishedJob(repoPath); if (!job) { process.stderr.write( - id ? `No job \`${id}\` for this repo.\n` : 'No finished open-agents for this repo yet.\n', + id + ? `No job \`${id}\` for this repo.\n` + : "No finished open-agents for this repo yet.\n", ); process.exit(1); } - if (job.status === 'running') { - process.stdout.write(`Job ${job.id} still running. Block with: open-agents wait ${job.id}\n`); + if (job.status === "running") { + process.stdout.write( + `Job ${job.id} still running. Block with: open-agents wait ${job.id}\n`, + ); process.exit(0); } process.stderr.write( - `open-agent ${job.id} — ${job.status} (exit ${job.exitCode ?? '?'})` + - (job.resultPath ? ` · ${job.resultPath}` : '') + - '\n', + `open-agent ${job.id} — ${job.status} (exit ${job.exitCode ?? "?"})` + + (job.resultPath ? ` · ${job.resultPath}` : "") + + "\n", ); process.stdout.write(readDeliverable(adapter, job)); - process.exit(job.status === 'done' ? 0 : 1); + process.exit(job.status === "done" ? 0 : 1); } function cmdResult(flags: Flags, positional: string[]): void { - printJobResult(pickAdapter(flags), flags.cwd || process.cwd(), positional[0] || null); + printJobResult( + pickAdapter(flags), + flags.cwd || process.cwd(), + positional[0] || null, + ); } // --- ls -------------------------------------------------------------------- @@ -500,17 +705,17 @@ function cmdLs(flags: Flags): void { const repoPath = flags.cwd || process.cwd(); const jobs = listJobs(repoPath, { limit: 20 }); if (!jobs.length) { - process.stdout.write('(no open-agents for this repo)\n'); + process.stdout.write("(no open-agents for this repo)\n"); return; } const now = Date.now(); for (const j of jobs as any[]) { const ageMin = Math.round((now - new Date(j.startedAt).getTime()) / 60000); const age = ageMin < 60 ? `${ageMin}m` : `${Math.round(ageMin / 60)}h`; - const tag = j.edit ? 'EDIT' : 'explore'; + const tag = j.edit ? "EDIT" : "explore"; process.stdout.write( `${j.id} ${j.status.padEnd(9)} ${tag.padEnd(5)} ${age.padStart(4)} ` + - `${j.prompt.replace(/\s+/g, ' ').slice(0, 70)}\n`, + `${j.prompt.replace(/\s+/g, " ").slice(0, 70)}\n`, ); } } @@ -529,23 +734,25 @@ async function cmdTail(flags: Flags, positional: string[]): Promise { let offset = 0; for (;;) { if (existsSync(job.rawLogPath)) { - const txt = readFileSync(job.rawLogPath, 'utf8'); + const txt = readFileSync(job.rawLogPath, "utf8"); if (txt.length > offset) { - for (const line of txt.slice(offset).split('\n')) { + for (const line of txt.slice(offset).split("\n")) { if (!line.trim()) continue; const ev = adapter.parseLine(line); if (!ev) { - process.stdout.write(line + '\n'); + process.stdout.write(line + "\n"); continue; } - const t = (ev as any).type || '?'; + const t = (ev as any).type || "?"; const txtBit = - (typeof (ev as any).text === 'string' && (ev as any).text) || + (typeof (ev as any).text === "string" && (ev as any).text) || ((ev as any).message && - typeof (ev as any).message.text === 'string' && + typeof (ev as any).message.text === "string" && (ev as any).message.text) || - ''; - process.stdout.write(`[${t}] ${String(txtBit).replace(/\s+/g, ' ').slice(0, 160)}\n`); + ""; + process.stdout.write( + `[${t}] ${String(txtBit).replace(/\s+/g, " ").slice(0, 160)}\n`, + ); } offset = txt.length; } @@ -560,7 +767,8 @@ async function cmdTail(flags: Flags, positional: string[]): Promise { // --- dispatch -------------------------------------------------------------- const HELP = `open-agents cli — self-waiting subagent job primitive - submit --file a.md [--file b.md …] | "" [--edit] [--model ] [--provider p] [--timeout s] [--cwd d] + submit --task "" | --spec-file a.md [--spec-file b.md …] | "" + [--no-history] [--edit] [--model ] [--provider p] [--timeout s] [--cwd d] --model : use a specific provider model for this submission. Overrides defaultModel in config and the provider's built-in default. --provider cursor (default, cursor-agent, default cursor-grok-4.6-high-fast) @@ -572,23 +780,24 @@ const HELP = `open-agents cli — self-waiting subagent job primitive under a "--- deliverable ---" divider, so one read sees everything. Use --no-inline to keep stdout to the compact header + file paths only (the old behavior — useful for scripts that just parse status). - many --file ⇒ run in parallel. --background (alias --detach, --bg, + --task and --spec-file may repeat; many inputs run in parallel. + --file remains an alias for --spec-file. + --background (alias --detach, --bg, --no-wait): print bare id(s), don't wait, no inline output. --parallel-mode asap|all-together (default asap): asap prints each index line the moment its job finishes; all-together = submit order. - --compact-mode: treat the input as a very short task title - (roughly 5-10 words) and ask a Cursor spec-writer using the - built-in default model to distill recent history into a concise - executor spec. Do not paste - details, quotes, or file lists into the inline prompt when the - conversation already contains that context. - Uses --history-file , OPEN_AGENTS_HISTORY_FILE, - CLAUDE_CODE_TRANSCRIPT_PATH, the current Claude Code transcript - when CLAUDE_CODE_SESSION_ID is set, CODEX_THREAD_ID sessions under - ~/.codex/sessions, or ~/.codex/history.jsonl. Gives the spec-writer - the last 50000 token-like units plus a pointer to the full history. - Progress output includes spec generation time and the generated - spec artifact path. + current-session conversation history is included automatically. + Up to 100000 token-like units are handed to the executor directly. + Larger histories use one curator run to select a separate relevant + context for each foreground task; the original task stays verbatim. + Exact sessions resolve from --history-file , + OPEN_AGENTS_HISTORY_FILE, CLAUDE_CODE_TRANSCRIPT_PATH / + CLAUDE_CODE_SESSION_ID, CODEX_TRANSCRIPT_PATH / CODEX_THREAD_ID, + or explicit Cursor transcript variables. There is no newest-file + fallback. If no exact session is available, submission continues + task-only with a warning. --no-history disables history explicitly. + --include-relevant-history and --compact-mode remain compatibility + aliases for the default history behavior. wait wait for ALL running jobs in this repo wait "" | wait for ALL of these wait --any return when ≥1 done; prints which (loop = popcorn) @@ -598,8 +807,8 @@ const HELP = `open-agents cli — self-waiting subagent job primitive exit codes: 0 ok · 1 job failed · 2 bad id/spec · 3 wait timeout common — ONE backgrounded, harness-tracked command (blocks, prints result(s)): - explore submit --file spec.md - explore submit --file a.md --file b.md --file c.md # parallel + explore submit --task "review LiveKit realtime architecture" + explore submit --spec-file a.md --spec-file b.md --spec-file c.md # parallel `; export async function main(argvRest?: string[]): Promise { @@ -607,33 +816,33 @@ export async function main(argvRest?: string[]): Promise { const { flags, positional } = parseFlags(rest); if (flags.unknownOptions?.length) { process.stderr.write( - `open-agents: unsupported option(s): ${flags.unknownOptions.join(', ')}. See --help.\n`, + `open-agents: unsupported option(s): ${flags.unknownOptions.join(", ")}. See --help.\n`, ); process.exit(2); } switch (verb) { - case '__run': + case "__run": return cmdRun(positional, flags); - case 'submit': + case "submit": return cmdSubmit(flags, positional); - case 'wait': + case "wait": return cmdWait(flags, positional); - case 'result': + case "result": return cmdResult(flags, positional); - case 'ls': + case "ls": return cmdLs(flags); - case 'tail': + case "tail": return cmdTail(flags, positional); - case 'check': { + case "check": { const code = await runCheck({ json: flags.json, cwd: flags.cwd, }); process.exit(code); } - case '--help': - case '-h': - case 'help': + case "--help": + case "-h": + case "help": process.stdout.write(HELP); process.exit(0); default: diff --git a/packages/open-agents/src/core/compact.ts b/packages/open-agents/src/core/compact.ts deleted file mode 100644 index 7d73293..0000000 --- a/packages/open-agents/src/core/compact.ts +++ /dev/null @@ -1,387 +0,0 @@ -import { existsSync, readFileSync, readdirSync, statSync } from 'node:fs'; -import { homedir } from 'node:os'; -import { join, resolve } from 'node:path'; -import { compact } from '@davstack/context-compactor'; - -export type CompactMessage = Record & { - role?: string; - content?: unknown; -}; - -export interface CompactHistory { - path: string; - text: string; - tail: string; - tailTokens: number; - messages: CompactMessage[]; -} - -export function tokenTail(text: string, maxTokens: number): string { - if (!Number.isFinite(maxTokens) || maxTokens <= 0) return ''; - const parts = text.match(/\s+|[A-Za-z0-9_]+|[^\sA-Za-z0-9_]/g) || []; - let tokens = 0; - let start = parts.length; - while (start > 0 && tokens < maxTokens) { - start -= 1; - if (!/^\s+$/.test(parts[start])) tokens += 1; - } - return parts.slice(start).join('').trim(); -} - -function tokenLikeCount(text: string): number { - return (text.match(/[A-Za-z0-9_]+|[^\sA-Za-z0-9_]/g) || []).length; -} - -function textFromContent(content: unknown): string { - if (typeof content === 'string') return content; - if (content == null) return ''; - if (Array.isArray(content)) { - return content - .map((part) => { - if (typeof part === 'string') return part; - if (!part || typeof part !== 'object') return ''; - const item = part as Record; - if (typeof item.text === 'string') return item.text; - if (typeof item.content === 'string') return item.content; - return JSON.stringify(item); - }) - .filter(Boolean) - .join('\n'); - } - return JSON.stringify(content); -} - -function compactMessageText(message: CompactMessage): string { - const role = typeof message.role === 'string' ? message.role : 'message'; - const content = textFromContent(message.content); - return `${role}: ${content || JSON.stringify(message)}`; -} - -export function renderCompactMessages(messages: CompactMessage[]): string { - return messages.map(compactMessageText).join('\n\n').trim(); -} - -function compactText(text: string): string { - if (!text) return text; - try { - return compact(text).text; - } catch { - return text; - } -} - -export function compactMessageContent(message: CompactMessage): CompactMessage { - const content = message.content; - if (typeof content === 'string') { - return { ...message, content: compactText(content) }; - } - if (Array.isArray(content)) { - const blocks = content.map((part) => { - if (!part || typeof part !== 'object') return part; - const item = part as Record; - if (typeof item.text === 'string') return { ...item, text: compactText(item.text) }; - if (typeof item.content === 'string') return { ...item, content: compactText(item.content) }; - return part; - }); - return { ...message, content: blocks }; - } - return { ...message }; -} - -export function compactMessages(messages: CompactMessage[]): { - messages: CompactMessage[]; - tokensBefore: number; - tokensAfter: number; - tokensSaved: number; -} { - const out: CompactMessage[] = []; - let tokensBefore = 0; - let tokensAfter = 0; - for (const message of messages) { - tokensBefore += tokenLikeCount(compactMessageText(message)); - const compacted = compactMessageContent(message); - tokensAfter += tokenLikeCount(compactMessageText(compacted)); - out.push(compacted); - } - return { messages: out, tokensBefore, tokensAfter, tokensSaved: tokensBefore - tokensAfter }; -} - -function isMessage(value: unknown): value is CompactMessage { - if (!value || typeof value !== 'object') return false; - const item = value as Record; - return typeof item.role === 'string' && 'content' in item; -} - -function extractMessage(row: unknown): CompactMessage | null { - if (isMessage(row)) return row; - if (!row || typeof row !== 'object') return null; - const item = row as Record; - if (isMessage(item.message)) return item.message; - if ( - typeof item.type === 'string' && - ['system', 'user', 'assistant', 'tool'].includes(item.type) && - 'content' in item - ) { - return { ...item, role: item.type }; - } - return null; -} - -export function parseCompactMessages(text: string): CompactMessage[] { - const messages: CompactMessage[] = []; - for (const line of text.split(/\r?\n/)) { - if (!line.trim()) continue; - try { - const message = extractMessage(JSON.parse(line)); - if (message) messages.push(message); - } catch { - return []; - } - } - return messages; -} - -export function tailCompactMessages( - messages: CompactMessage[], - maxTokens: number, -): CompactMessage[] { - if (!Number.isFinite(maxTokens) || maxTokens <= 0) return []; - const out: CompactMessage[] = []; - let tokens = 0; - for (let i = messages.length - 1; i >= 0; i -= 1) { - const messageTokens = tokenLikeCount(compactMessageText(messages[i])); - if (out.length && tokens + messageTokens > maxTokens) break; - out.unshift(messages[i]); - tokens += messageTokens; - if (tokens >= maxTokens) break; - } - return out; -} - -export function findClaudeTranscriptBySession( - sessionId: string, - homeDir = homedir(), -): string | null { - if (!sessionId.trim()) return null; - const projectsDir = join(homeDir, '.claude', 'projects'); - if (!existsSync(projectsDir)) return null; - for (const entry of readdirSync(projectsDir, { withFileTypes: true })) { - if (!entry.isDirectory()) continue; - const candidate = join(projectsDir, entry.name, `${sessionId}.jsonl`); - if (existsSync(candidate)) return candidate; - } - return null; -} - -function walkFiles(dir: string, predicate: (path: string) => boolean, limit = 5000): string[] { - const out: string[] = []; - const visit = (current: string) => { - if (out.length >= limit) return; - let entries; - try { - entries = readdirSync(current, { withFileTypes: true }); - } catch { - return; - } - for (const entry of entries) { - if (out.length >= limit) return; - const full = join(current, entry.name); - if (entry.isDirectory()) visit(full); - else if (entry.isFile() && predicate(full)) out.push(full); - } - }; - if (existsSync(dir)) visit(dir); - return out; -} - -function newest(paths: string[]): string | null { - return ( - paths - .map((path) => { - try { - return { path, mtimeMs: statSync(path).mtimeMs }; - } catch { - return null; - } - }) - .filter((x): x is { path: string; mtimeMs: number } => x !== null) - .sort((a, b) => b.mtimeMs - a.mtimeMs)[0]?.path ?? null - ); -} - -export function findCodexTranscriptBySession( - sessionId: string, - homeDir = homedir(), -): string | null { - if (!sessionId.trim()) return null; - const sessionsDir = join(homeDir, '.codex', 'sessions'); - return newest( - walkFiles( - sessionsDir, - (path) => path.endsWith('.jsonl') && path.includes(sessionId), - ), - ); -} - -export function findLatestCodexHistory(homeDir = homedir()): string | null { - const codexDir = join(homeDir, '.codex'); - const history = join(codexDir, 'history.jsonl'); - if (existsSync(history)) return history; - return newest(walkFiles(join(codexDir, 'sessions'), (path) => path.endsWith('.jsonl'))); -} - -export function findLatestCursorAgentHistory(homeDir = homedir()): string | null { - const explicitRoots = [ - join(homeDir, 'AppData', 'Roaming', 'Cursor'), - join(homeDir, 'AppData', 'Local', 'cursor-agent'), - join(homeDir, '.cursor-agent'), - ]; - for (const root of explicitRoots) { - const hit = newest( - walkFiles(root, (path) => /(?:history|session|transcript|conversation).*\.jsonl$/i.test(path), 2000), - ); - if (hit) return hit; - } - return null; -} - -export function resolveCompactHistoryFile(input?: { - historyFile?: string; - env?: NodeJS.ProcessEnv; - homeDir?: string; -}): string | null { - const env = input?.env ?? process.env; - const explicit = input?.historyFile || env.OPEN_AGENTS_HISTORY_FILE; - if (explicit) return resolve(explicit); - - const transcriptPath = env.CLAUDE_CODE_TRANSCRIPT_PATH; - if (transcriptPath) return resolve(transcriptPath); - - const sessionId = env.CLAUDE_CODE_SESSION_ID; - if (sessionId) return findClaudeTranscriptBySession(sessionId, input?.homeDir); - - const codexTranscriptPath = env.CODEX_TRANSCRIPT_PATH || env.CODEX_SESSION_FILE; - if (codexTranscriptPath) return resolve(codexTranscriptPath); - - const codexThreadId = env.CODEX_THREAD_ID || env.CODEX_SESSION_ID; - if (codexThreadId) { - const codexTranscript = findCodexTranscriptBySession(codexThreadId, input?.homeDir); - if (codexTranscript) return codexTranscript; - } - - const cursorTranscriptPath = - env.CURSOR_AGENT_TRANSCRIPT_PATH || env.CURSOR_AGENT_HISTORY_FILE || env.AGENT_HISTORY_FILE; - if (cursorTranscriptPath) return resolve(cursorTranscriptPath); - - const cursorHistory = findLatestCursorAgentHistory(input?.homeDir); - if (cursorHistory) return cursorHistory; - - return findLatestCodexHistory(input?.homeDir); -} - -export function loadCompactHistory(path: string, tailTokens: number): CompactHistory { - const resolved = resolve(path); - if (!existsSync(resolved)) { - throw new Error(`compact history file not found: ${resolved}`); - } - const text = readFileSync(resolved, 'utf8'); - const parsed = parseCompactMessages(text); - const compacted = parsed.length ? compactMessages(parsed).messages : parsed; - const messages = tailCompactMessages(compacted, tailTokens); - return { - path: resolved, - text, - tail: messages.length ? renderCompactMessages(messages) : tokenTail(text, tailTokens), - tailTokens, - messages, - }; -} - -export function buildCompactSpecWriterTask(input: { - task: string; - repoPath: string; - targetProfile: string; - historyPath: string; - historyTail: string; - tailTokens: number; -}): string { - return `# Compact Mode Spec Handoff - -You are the spec-writer agent in a two-agent handoff. - -The orchestrating user intentionally provided only a short task title. Use the -recent history tail below to recover the detailed context needed by the executor. - -Short task: -${input.task} - -Repository path: -${input.repoPath} - -Executor profile: -${input.targetProfile} - -Full history pointer: -${input.historyPath} - -Write a compact execution spec for the executor agent. Preserve the user's -latest intent, scope, constraints, preferences, and non-goals. Do not solve the -task yourself. Do not invent requirements that are not supported by the short -task or history tail. - -Keep the generated spec super concise and brief. Prefer concise bullets over -prose. Do not enumerate every remembered detail; include only what changes the -executor's behavior. The full history pointer is available, so if context is -uncertain or too detailed to summarize cleanly, point the executor to inspect -the relevant part of the history instead of copying it into the spec. - -Strongly prioritize the block, but keep it distilled. The executor -should receive a clean brief, not the transcript or full history tail. Include -dense bullet points with only the history details that could help the executor -succeed, including: - -- relevant files, folders, packages, commands, flags, and artifact paths; -- the user's original query and short direct quotes when they clarify intent; -- project-specific facts and conventions; -- decisions already made in the conversation; -- constraints, non-goals, risks, and caveats; -- details that may look incidental but could matter during implementation. - -Do not paste the full history, full recent tail, or long conversation excerpts -into the generated spec. Use short direct quotes only when they clarify intent. -Omit irrelevant turns, dead ends, and stale decisions that the user's latest -messages supersede. - -Do not add an block unless the history explicitly asks for one. -For compact handoffs, a rich block is more important than a formal -acceptance checklist. - -Return only the generated execution spec, preferably using this shape: - - -... - - - -- ... -- ... - - - -... - - - -... - - -Recent history tail: -The following block contains the last ${input.tailTokens} token-like units from -the provided history file. This block is intentionally placed at the bottom of -the prompt so the newest relevant context is closest to generation. - - -${input.historyTail || '(empty)'} - -`; -} diff --git a/packages/open-agents/src/core/history.ts b/packages/open-agents/src/core/history.ts new file mode 100644 index 0000000..04ada74 --- /dev/null +++ b/packages/open-agents/src/core/history.ts @@ -0,0 +1,310 @@ +import { existsSync, readFileSync, readdirSync, statSync } from "node:fs"; +import { homedir } from "node:os"; +import { join, resolve } from "node:path"; + +export type ConversationMessage = { + role: "user" | "assistant"; + content: string; +}; + +export const DEFAULT_DIRECT_HISTORY_TOKENS = 100_000; + +export interface ConversationHistory { + path: string; + text: string; + tokens: number; + messages: ConversationMessage[]; +} + +export interface CuratedHistory { + taskId: string; + relevantHistory: string; +} + +export function tokenLikeCount(text: string): number { + return (text.match(/[A-Za-z0-9_]+|[^\sA-Za-z0-9_]/g) || []).length; +} + +function visibleTextFromContent(content: unknown): string { + if (typeof content === "string") return content.trim(); + if (!Array.isArray(content)) return ""; + return content + .map((part) => { + if (typeof part === "string") return part; + if (!part || typeof part !== "object") return ""; + const item = part as Record; + const type = typeof item.type === "string" ? item.type : ""; + if (type && !type.includes("text")) return ""; + if (typeof item.text === "string") return item.text; + if (typeof item.content === "string") return item.content; + return ""; + }) + .filter(Boolean) + .join("\n") + .trim(); +} + +function genericVisibleMessage(row: unknown): ConversationMessage | null { + if (!row || typeof row !== "object") return null; + const item = row as Record; + const candidate = + item.message && typeof item.message === "object" + ? (item.message as Record) + : item; + const role = candidate.role; + if (role !== "user" && role !== "assistant") return null; + const content = visibleTextFromContent(candidate.content); + return content ? { role, content } : null; +} + +function codexVisibleMessage(row: unknown): ConversationMessage | null { + if (!row || typeof row !== "object") return null; + const item = row as Record; + if ( + item.type !== "event_msg" || + !item.payload || + typeof item.payload !== "object" + ) { + return null; + } + const payload = item.payload as Record; + const content = payload.message ?? payload.text; + if (typeof content !== "string" || !content.trim()) return null; + if (payload.type === "user_message") { + return { role: "user", content: content.trim() }; + } + if (payload.type === "agent_message") { + return { role: "assistant", content: content.trim() }; + } + return null; +} + +export function parseConversationMessages(text: string): ConversationMessage[] { + const codexMessages: ConversationMessage[] = []; + const genericMessages: ConversationMessage[] = []; + for (const line of text.split(/\r?\n/)) { + if (!line.trim()) continue; + let row: unknown; + try { + row = JSON.parse(line); + } catch { + continue; + } + const codexMessage = codexVisibleMessage(row); + if (codexMessage) { + codexMessages.push(codexMessage); + continue; + } + const genericMessage = genericVisibleMessage(row); + if (genericMessage) genericMessages.push(genericMessage); + } + return codexMessages.length ? codexMessages : genericMessages; +} + +export function renderConversationMessages( + messages: ConversationMessage[], +): string { + return messages + .map( + (message) => + `${message.role === "assistant" ? "Assistant" : "User"}:\n${message.content}`, + ) + .join("\n\n") + .trim(); +} + +export function loadConversationHistory(path: string): ConversationHistory { + const resolved = resolve(path); + if (!existsSync(resolved)) { + throw new Error(`history file not found: ${resolved}`); + } + const messages = parseConversationMessages(readFileSync(resolved, "utf8")); + const text = renderConversationMessages(messages); + return { path: resolved, text, tokens: tokenLikeCount(text), messages }; +} + +export function historyNeedsCuration( + historyTokens: number, + directLimit = DEFAULT_DIRECT_HISTORY_TOKENS, +): boolean { + return historyTokens > directLimit; +} + +export function buildHistoryCuratorTask(input: { + tasks: Array<{ taskId: string; task: string }>; + historyPath: string; + historyText: string; +}): string { + return `# Select task-relevant conversation history + +The executor tasks below are authoritative and must remain verbatim. For each +task, select only prior conversation details that could change how its executor +works: decisions, constraints, corrections, non-goals, useful paths, and +unresolved questions. Do not solve, expand, or rewrite any task. Do not create +one shared summary: relevance is task-specific. + +Return strict JSON only, with exactly one entry for every supplied task ID: +{"contexts":[{"taskId":"task-1","relevantHistory":"concise relevant context"}]} + +Keep each relevantHistory value concise. Omit unrelated turns, tool chatter, +status updates, stale decisions superseded later, and instructions unrelated to +the task. If nothing is relevant, use an empty string. + +History source: ${input.historyPath} + +Tasks: +${JSON.stringify(input.tasks, null, 2)} + +Conversation history: + +${input.historyText} + +`; +} + +export function parseCuratedHistory( + text: string, + taskIds: string[], +): Map { + const start = text.indexOf("{"); + const end = text.lastIndexOf("}"); + if (start < 0 || end < start) { + throw new Error("history curator returned no JSON object"); + } + const parsed = JSON.parse(text.slice(start, end + 1)) as { + contexts?: CuratedHistory[]; + }; + if (!Array.isArray(parsed.contexts)) { + throw new Error("history curator JSON is missing contexts"); + } + const expected = new Set(taskIds); + const contexts = new Map(); + for (const item of parsed.contexts) { + if ( + !item || + typeof item.taskId !== "string" || + typeof item.relevantHistory !== "string" + ) { + throw new Error("history curator returned an invalid context entry"); + } + if (!expected.has(item.taskId) || contexts.has(item.taskId)) { + throw new Error( + `history curator returned an unexpected or duplicate task ID: ${item.taskId}`, + ); + } + contexts.set(item.taskId, item.relevantHistory.trim()); + } + const missing = taskIds.filter((taskId) => !contexts.has(taskId)); + if (missing.length) { + throw new Error(`history curator omitted task IDs: ${missing.join(", ")}`); + } + return contexts; +} + +export function findClaudeTranscriptBySession( + sessionId: string, + homeDir = homedir(), +): string | null { + if (!sessionId.trim()) return null; + const projectsDir = join(homeDir, ".claude", "projects"); + if (!existsSync(projectsDir)) return null; + for (const entry of readdirSync(projectsDir, { withFileTypes: true })) { + if (!entry.isDirectory()) continue; + const candidate = join(projectsDir, entry.name, `${sessionId}.jsonl`); + if (existsSync(candidate)) return candidate; + } + return null; +} + +function walkFiles( + dir: string, + predicate: (path: string) => boolean, + limit = 5000, +): string[] { + const out: string[] = []; + const visit = (current: string) => { + if (out.length >= limit) return; + let entries; + try { + entries = readdirSync(current, { withFileTypes: true }); + } catch { + return; + } + for (const entry of entries) { + if (out.length >= limit) return; + const full = join(current, entry.name); + if (entry.isDirectory()) visit(full); + else if (entry.isFile() && predicate(full)) out.push(full); + } + }; + if (existsSync(dir)) visit(dir); + return out; +} + +function newest(paths: string[]): string | null { + return ( + paths + .map((path) => { + try { + return { path, mtimeMs: statSync(path).mtimeMs }; + } catch { + return null; + } + }) + .filter( + (item): item is { path: string; mtimeMs: number } => item !== null, + ) + .sort((a, b) => b.mtimeMs - a.mtimeMs)[0]?.path ?? null + ); +} + +export function findCodexTranscriptBySession( + sessionId: string, + homeDir = homedir(), +): string | null { + if (!sessionId.trim()) return null; + return newest( + walkFiles( + join(homeDir, ".codex", "sessions"), + (path) => path.endsWith(".jsonl") && path.includes(sessionId), + ), + ); +} + +export function resolveConversationHistoryFile(input?: { + historyFile?: string; + env?: NodeJS.ProcessEnv; + homeDir?: string; +}): string | null { + const env = input?.env ?? process.env; + const explicit = input?.historyFile || env.OPEN_AGENTS_HISTORY_FILE; + if (explicit) return resolve(explicit); + + if (env.CLAUDE_CODE_TRANSCRIPT_PATH) { + return resolve(env.CLAUDE_CODE_TRANSCRIPT_PATH); + } + if (env.CLAUDE_CODE_SESSION_ID) { + return findClaudeTranscriptBySession( + env.CLAUDE_CODE_SESSION_ID, + input?.homeDir, + ); + } + + const codexTranscriptPath = + env.CODEX_TRANSCRIPT_PATH || env.CODEX_SESSION_FILE; + if (codexTranscriptPath) return resolve(codexTranscriptPath); + const codexThreadId = env.CODEX_THREAD_ID || env.CODEX_SESSION_ID; + if (codexThreadId) { + const transcript = findCodexTranscriptBySession( + codexThreadId, + input?.homeDir, + ); + if (transcript) return transcript; + } + + const cursorTranscriptPath = + env.CURSOR_AGENT_TRANSCRIPT_PATH || + env.CURSOR_AGENT_HISTORY_FILE || + env.AGENT_HISTORY_FILE; + return cursorTranscriptPath ? resolve(cursorTranscriptPath) : null; +} diff --git a/packages/open-agents/src/core/jobs.ts b/packages/open-agents/src/core/jobs.ts index 64d166f..e242b55 100644 --- a/packages/open-agents/src/core/jobs.ts +++ b/packages/open-agents/src/core/jobs.ts @@ -9,11 +9,11 @@ import { statSync, unlinkSync, writeFileSync, -} from 'node:fs'; -import { join } from 'node:path'; -import { ensureDir, jobsDir, logsDir } from './paths.js'; +} from "node:fs"; +import { join } from "node:path"; +import { ensureDir, jobsDir, logsDir } from "./paths.js"; -export type JobStatus = 'running' | 'done' | 'failed' | 'cancelled'; +export type JobStatus = "running" | "done" | "failed" | "cancelled"; export interface JobRecord { id: string; @@ -35,6 +35,13 @@ export interface JobRecord { fullPrompt?: string; edit?: boolean; timeoutSec?: number; + adapterName?: string; + compactTask?: string; + compactHistoryFile?: string; + historyTask?: string; + historySourceFile?: string; + historyMode?: "none" | "direct" | "curated"; + historyTokens?: number; resultPath?: string; killed?: boolean; } @@ -63,7 +70,7 @@ export function rawLogPath(repoPath: string, id: string): string { function atomicWrite(target: string, data: string): void { const tmp = `${target}.tmp-${process.pid}-${Date.now()}`; - writeFileSync(tmp, data, 'utf8'); + writeFileSync(tmp, data, "utf8"); renameSync(tmp, target); } @@ -75,13 +82,16 @@ export function createJob(init: CreateJobInit): JobRecord { repoPath: init.repoPath, prompt: init.prompt, model: init.model, - status: 'running', + status: "running", startedAt: new Date().toISOString(), rawLogPath: rawLogPath(init.repoPath, init.id), ...(init.background ? { background: true } : {}), ...(init.cloud ? { cloud: true } : {}), }; - atomicWrite(jobFilePath(init.repoPath, init.id), JSON.stringify(record, null, 2)); + atomicWrite( + jobFilePath(init.repoPath, init.id), + JSON.stringify(record, null, 2), + ); return record; } @@ -89,9 +99,9 @@ export function readJob(repoPath: string, id: string): JobRecord | null { const file = jobFilePath(repoPath, id); if (!existsSync(file)) return null; try { - const raw = readFileSync(file, 'utf8'); + const raw = readFileSync(file, "utf8"); const parsed = JSON.parse(raw); - if (parsed && typeof parsed === 'object' && typeof parsed.id === 'string') { + if (parsed && typeof parsed === "object" && typeof parsed.id === "string") { return parsed as JobRecord; } return null; @@ -115,13 +125,19 @@ export function updateJob( export function listJobs(repoPath: string, opts: ListOpts = {}): JobRecord[] { const dir = jobsDir(repoPath); if (!existsSync(dir)) return []; - const files = readdirSync(dir).filter((f) => f.endsWith('.json') && !f.endsWith('.tmp')); + const files = readdirSync(dir).filter( + (f) => f.endsWith(".json") && !f.endsWith(".tmp"), + ); const records: JobRecord[] = []; for (const f of files) { try { - const raw = readFileSync(join(dir, f), 'utf8'); + const raw = readFileSync(join(dir, f), "utf8"); const parsed = JSON.parse(raw); - if (parsed && typeof parsed === 'object' && typeof parsed.id === 'string') { + if ( + parsed && + typeof parsed === "object" && + typeof parsed.id === "string" + ) { records.push(parsed as JobRecord); } } catch { @@ -129,8 +145,12 @@ export function listJobs(repoPath: string, opts: ListOpts = {}): JobRecord[] { } } records.sort((a, b) => (a.startedAt < b.startedAt ? 1 : -1)); - const filtered = opts.status ? records.filter((r) => r.status === opts.status) : records; - return typeof opts.limit === 'number' ? filtered.slice(0, opts.limit) : filtered; + const filtered = opts.status + ? records.filter((r) => r.status === opts.status) + : records; + return typeof opts.limit === "number" + ? filtered.slice(0, opts.limit) + : filtered; } export function pruneOlderThanDays(repoPath: string, days = 30): number { @@ -181,10 +201,10 @@ export async function cancelJob( ): Promise { const job = readJob(repoPath, id); if (!job) return null; - if (job.status !== 'running') return job; - if (typeof job.pid === 'number' && isProcessAlive(job.pid)) { + if (job.status !== "running") return job; + if (typeof job.pid === "number" && isProcessAlive(job.pid)) { try { - process.kill(job.pid, 'SIGTERM'); + process.kill(job.pid, "SIGTERM"); } catch { // may have exited } @@ -194,23 +214,23 @@ export async function cancelJob( } if (isProcessAlive(job.pid)) { try { - process.kill(job.pid, 'SIGKILL'); + process.kill(job.pid, "SIGKILL"); } catch { // ignore } } } return updateJob(repoPath, id, { - status: 'cancelled', + status: "cancelled", finishedAt: new Date().toISOString(), }); } export function findRunningJobs(repoPath: string): JobRecord[] { - return listJobs(repoPath).filter((j) => j.status === 'running'); + return listJobs(repoPath).filter((j) => j.status === "running"); } export function mostRecentFinishedJob(repoPath: string): JobRecord | null { - const jobs = listJobs(repoPath).filter((j) => j.status !== 'running'); + const jobs = listJobs(repoPath).filter((j) => j.status !== "running"); return jobs[0] ?? null; } diff --git a/packages/open-agents/src/profiles/history-curator.ts b/packages/open-agents/src/profiles/history-curator.ts new file mode 100644 index 0000000..f1c7bbf --- /dev/null +++ b/packages/open-agents/src/profiles/history-curator.ts @@ -0,0 +1,17 @@ +import { type Profile, assembleScaffold } from "./types.js"; + +const GUARDS = + "- HISTORY CURATOR ONLY: select task-relevant facts from the supplied conversation. Do not solve or rewrite the tasks.\n" + + "- READ-ONLY: do not modify repository files.\n" + + "- Keep every task isolated. Never merge their context or copy context between unrelated tasks.\n" + + "- OUTPUT: strict JSON only, matching the schema in the supplied task. No markdown fences or commentary.\n"; + +export const historyCuratorProfile: Profile = { + name: "history-curator", + tag: "history", + mode: "ask", + buildPrompt(specBody: string, addendum?: string) { + return assembleScaffold(specBody, GUARDS, addendum); + }, + warnIfMissingAcceptance() {}, +}; diff --git a/packages/open-agents/src/profiles/spec-writer.ts b/packages/open-agents/src/profiles/spec-writer.ts deleted file mode 100644 index 1a6343d..0000000 --- a/packages/open-agents/src/profiles/spec-writer.ts +++ /dev/null @@ -1,22 +0,0 @@ -import { type Profile, assembleScaffold } from './types.js'; - -const GUARDS = - '- SPEC WRITER ONLY: write a compact execution spec for a later subagent. Do not solve the task.\n' + - '- READ-ONLY: inspect files only if the prompt explicitly asks you to use repository context.\n' + - '- Preserve concrete user intent, the original user query, short useful quotes, constraints, file scope, project context, conversation decisions, and non-goals from the history.\n' + - '- Distill the history. Do not copy the transcript, full history tail, or long excerpts into the generated spec.\n' + - '- Keep the generated spec super concise and brief; point to the history file for uncertain details instead of copying them.\n' + - '- Do not invent requirements. If important context is missing, state the ambiguity in the spec.\n' + - '- OUTPUT: only the generated execution spec. Use , , , and tags. Do not include an block unless the history explicitly requires one.\n'; - -export const specWriterProfile: Profile = { - name: 'spec-writer', - tag: 'spec', - mode: 'ask', - buildPrompt(specBody: string, addendum?: string) { - return assembleScaffold(specBody, GUARDS, addendum); - }, - warnIfMissingAcceptance() { - // The spec writer produces acceptance criteria; its own prompt does not need them. - }, -}; diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 9cd2e3f..85a5e6c 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -141,9 +141,6 @@ importers: '@davstack/cli-utils': specifier: workspace:* version: link:../cli-utils - '@davstack/context-compactor': - specifier: workspace:* - version: link:../context-compactor devDependencies: '@types/node': specifier: ^25.9.1 diff --git a/skills/explore/SKILL.md b/skills/explore/SKILL.md index 5fc9f37..43214a3 100644 --- a/skills/explore/SKILL.md +++ b/skills/explore/SKILL.md @@ -8,32 +8,43 @@ description: >- you wouldn't otherwise open. --- -Scope tightly, then run (backgrounded — the harness notifies you): +Default to one short `--task`, especially for lookups, traces, and other simple +requests. Use the user's request verbatim when it is already concise: - explore submit --file ~/.davstack/specs/.md +explore submit --task "Find where Juno's realtime voice is locked to marin, with exact path:line citations" -When the current conversation already contains the real context, prefer compact -mode over writing a spec: +Do not create a spec file or invent ``, ``, or `` merely +to wrap a task the user already stated. Conversation history supplies the +surrounding context automatically. - explore submit --compact-mode "audit supervisor handoffs" +Run one foreground submission with a long shell timeout. If the shell yields a +running handle, wait on that same handle; completion wakes the agent. Never pass +`--background`, poll `result`, or resubmit. -Keep the inline compact prompt to roughly 5-10 words. Do not paste details, -requirements, quotes, or file lists into it; the compact spec-writer reads the -current transcript/history and distills that context for the executor. Trust the -subagent handoff unless the task is genuinely too ambiguous from conversation -history. +Conversation history is included automatically. Keep the task to roughly one +sentence; do not repeat details, quotes, or file lists already present in the +conversation. History up to the direct budget goes to the executor unchanged; +oversized history is reduced to task-specific context first. Use `--no-history` +only when the conversation is irrelevant. `--compact-mode` remains a legacy +alias and is no longer needed. -For a **single scoped fact**, skip the spec file — inline it (no boilerplate): +If the conversation contains secrets, credentials, audit artifacts, or +user/health data, use `--no-history` and provide a sanitized scoped spec. - explore submit 'Exact signature + return type of resolve_query_adapter backend/src/query/adapter.py only' +For a single scoped fact, use `--task` with no boilerplate: -Many `--file` run in parallel from one command. Read the `result → ` -file for the answer. + explore submit --task "Find the exact signature and return type of resolve_query_adapter in backend/src/query/adapter.py" -Begin every `--file` spec with a markdown `# 3-5 word title` line — a short +Use `--spec-file` only when essential instructions not present in conversation +cannot fit cleanly in one sentence, or when supplying an intentionally prepared +multi-part spec. Never generate one pre-emptively. Multiple `--task` or +`--spec-file` inputs can run in parallel. + +If a spec file is genuinely needed, begin it with a markdown `# 3-5 word title` +line — a short overview of the task. The TUI agent viewer renders this as the job label; without it the viewer falls back to the first 5 words of the spec, which is -rarely meaningful. (Inline single-fact submits can skip the heading.) +rarely meaningful. The spec is just goal / context (the one gotcha) / scope tags. Do NOT add an output section — the structured `path:line` deliverable is automatic. diff --git a/skills/fast-edit/SKILL.md b/skills/fast-edit/SKILL.md index 8449b39..839361f 100644 --- a/skills/fast-edit/SKILL.md +++ b/skills/fast-edit/SKILL.md @@ -9,28 +9,41 @@ description: >- critical, complex edits that require careful judgement. --- -Run (backgrounded — the harness notifies you): +Default to one short `--task` whenever the mechanical edit can be stated in one +sentence, especially when conversation history already contains the details: - npx fast-edit submit --file ~/.davstack/specs/.md + fast-edit submit --task "Rename fooBar to computeFoo and update its callers without changing behavior" -When the current conversation already contains the real context, prefer compact -mode over writing a spec: +Do not create a spec file or invent intent/changes/constraints tags merely to +wrap a task the user already stated. Conversation history supplies the +surrounding context automatically. - npx fast-edit submit --compact-mode "rename legacy adapter" +Run one foreground submission with a long shell timeout. If the shell yields a +running handle, wait on that same handle; completion wakes the agent. Never pass +`--background`, poll `result`, or resubmit. -Keep the inline compact prompt to roughly 5-10 words. Do not paste details, -requirements, quotes, or file lists into it; the compact spec-writer reads the -current transcript/history and distills that context for the executor. Trust the -subagent handoff unless the edit is too risky or underspecified from -conversation history. +Conversation history is included automatically. Keep the task to roughly one +sentence; do not repeat details, quotes, or file lists already present in the +conversation. History up to the direct budget goes to the executor unchanged; +oversized history is reduced to task-specific context first. Use `--no-history` +only when the conversation is irrelevant. `--compact-mode` remains a legacy +alias and is no longer needed. -**Routing test.** Delegate when a *short* intent+constraints spec is enough +If the conversation contains secrets, credentials, audit artifacts, or +user/health data, use `--no-history` and provide a sanitized scoped spec. + +**Routing test.** Delegate when a short task is enough for the executor to produce the **full** intended edit. If writing the spec would mean pasting the new file contents or spelling out every line, the spec costs as much as the edit — just do it yourself. (Having read the files is fine; verbatim-detail specs are the only real waste.) -Begin every spec with a markdown `# 3-5 word title` line — a short overview of +Use `--spec-file` only when essential constraints not present in conversation +cannot fit cleanly in one sentence, or when supplying an intentionally prepared +multi-part spec. Never generate one pre-emptively. + +If a spec file is genuinely needed, begin it with a markdown `# 3-5 word title` +line — a short overview of the task. The TUI agent viewer renders this as the job label; without it the viewer falls back to the first 5 words of the spec, which is rarely meaningful.