feat(compaction): reuse prompt caches during compaction

docs/compaction.md

@@ -69,7 +69,7 @@ Completed and remaining work, active progress, and next action.
 Relevant files, commands, errors, and tool results.
 ```
 
-The shared prompt is built by `llm.BuildCompactionPrompt`. Automatic compaction adds an internal-checkpoint instruction through `llm.BuildAutoCompactionPrompt`.
+The shared `compactionGuidance` (in `internal/llm/systemprompt.go`) feeds both `llm.BuildCompactionPrompt` (manual) and `llm.BuildAutoCompactionPrompt` (automatic), each sent as the final user message after the normal request history. Both frame the request as a context compaction whose reply replaces the conversation history, ask the model never to use tools and to work from the existing conversation history alone, demand exact file paths, commands, identifiers, and error text with no references to the discarded history, and present the sections above as a baseline: extra sections and additional detail are allowed when the conversation calls for them. Both compaction paths enforce the no-tools guidance: a tool call the model attempts is rejected with a tool-error result instead of being executed.
 
 Automatic summaries additionally:
 
@@ -99,10 +99,10 @@ AppState snapshots persisted conversation history
         |
         v
 Build manual compaction request
-  - compaction system prompt
+  - normal agent system prompt
   - conversation snapshot
-  - final summarize instruction
-  - no tools
+  - final summarize instruction as a user message
+  - same tools retained; attempted tool calls rejected with a tool-error result
         |
         v
 Stream summary visibly in the REPL
@@ -123,11 +123,11 @@ Persist compaction_applied session event
 
 `AppState.StreamCompact` sends:
 
-1. a `RoleSystem` message containing `BuildCompactionPrompt(extraPrompt)`;
+1. the same `RoleSystem` message as a regular turn (`llm.Build` with current project instructions, skills, subagents, memory, and agent mode);
 2. a clone of current AppState messages;
-3. a final user instruction requesting the continuation summary.
+3. a final `RoleUser` message containing `BuildCompactionPrompt(extraPrompt)`.
 
-The request has no tools. Unlike automatic compaction, the summary is rendered as a normal visible stream.
+The request also sets `StreamOptions.DisableToolCalls` and `StreamOptions.DisableAutoCompaction`. Disabling tool calls rejects any tool the model attempts with a tool-error result — `Tool calls are disabled during compaction; use the history.` — emitted as a tool card in the transcript and returned to the model, which then continues from the existing history. At the execution step the client swaps in a denying tool registry whose tools accept any input but reject execution with that message, so no tool runs and no permission prompt appears. Disabling auto-compaction stops a nested automatic compaction from firing underneath the in-flight manual compaction (tool turns keep the stream running, so a threshold crossing is reachable). Keeping the system prompt, history, and tool definitions identical to the previous turn lets provider prompt caches (KV cache) reuse the conversation prefix instead of reprocessing it. The applied summary uses only the assistant text produced after the last tool activity, so pre-tool preamble is not folded into it. Unlike automatic compaction, the summary is rendered as a normal visible stream.
 
 ### Applying the result
 
@@ -209,21 +209,23 @@ Reduce old tool results if needed
 
 ### Private compaction request
 
-`llm.AutoCompact` creates a nested request using the same provider client:
+`llm.AutoCompact` creates a nested request using the same provider client. It preserves the current request as a cacheable prefix, appends `BuildAutoCompactionPrompt()` as the final user message, and retains the normal tool registry:
 
 ```go
-client.StreamChat(ctx, request, nil, llm.StreamOptions{
+client.StreamChat(ctx, request, toolRegistry, llm.StreamOptions{
     SessionID:             sessionID,
     OneShot:               true,
     DisableAutoCompaction: true,
+    DisableToolCalls:      true,
 })
 ```
 
 The nested request:
 
-- has no tools;
-- is one-shot;
-- disables automatic compaction to prevent recursive compaction;
+- keeps the normal system prompt, conversation history, and tool definitions for provider prompt-cache parity;
+- appends the compaction instruction as the final user message;
+- is one-shot and disables recursive automatic compaction;
+- rejects attempted tool calls with `Tool calls are disabled during compaction; use the history.` instead of executing them;
 - privately collects only assistant text and usage;
 - rejects an empty summary;
 - does not forward summary chunks, reasoning, or tool events to the parent stream.
@@ -385,9 +387,9 @@ The earlier assistant checkpoint remains available in the transcript for UI repl
 | Trigger | Explicit slash command | 90% proactive threshold or local hard-budget failure |
 | Runs inside parent turn | No | Yes |
 | Summary visibility | Visible | Private |
-| Tools in summary request | None | None |
+| Tools in summary request | Same registry; attempted calls rejected with a tool-error result | None |
 | Latest user message | Summarized with history | Retained verbatim outside summary |
-| System prompt during summary | Dedicated compaction prompt | Dedicated automatic compaction prompt |
+| System prompt during summary | Normal agent system prompt (reuses prompt cache) | Dedicated automatic compaction prompt |
 | Parent tools after compaction | Recreated on next normal turn | Preserved and reused immediately |
 | `Esc` behavior | Cancels manual compaction | Cancels only child compactor |
 | Replacement in AppState | One `RoleUser` summary | System-free automatic replacement |
@@ -399,9 +401,9 @@ The earlier assistant checkpoint remains available in the transcript for UI repl
 
 | Area | Files |
 |---|---|
-| Shared prompts and compactor | `internal/llm/systemprompt.go`, `internal/llm/auto_compaction.go` |
+| Shared prompts and compactor | `internal/llm/systemprompt.go`, `internal/llm/tool_execution.go`, `internal/llm/auto_compaction.go` |
 | Budgeting and context reduction | `internal/llm/context_reducer.go` |
-| Lifecycle event contract | `internal/llm/message.go`, `internal/llm/client.go` |
+| Lifecycle event contract | `internal/llm/core/message.go`, `internal/llm/client.go` |
 | Provider loops | `internal/llm/openai.go`, `openai_responses.go`, `openai_codex.go`, `anthropic.go`, `genkit.go`, `bedrock.go` |
 | Manual AppState flow | `internal/cli/repl/appstate/state.go`, `internal/cli/repl/command_handlers.go` |
 | Interactive automatic handling | `internal/cli/repl/handlers.go`, `internal/cli/repl/stream_handler.go` |

internal/cli/repl/appstate/state.go

@@ -13,8 +13,6 @@ import (
 	"github.com/mochow13/keen-code/internal/tools"
 )
 
-const compactionUserInstruction = "Please compact this conversation according to the system instructions."
-
 type AppState struct {
 	messages        []core.Message
 	llmClient       llm.LLMClient
@@ -180,12 +178,15 @@ func (s *AppState) StreamChat(ctx context.Context, cfg *config.ResolvedConfig, o
 	if s.llmClient == nil {
 		return nil, nil
 	}
-	systemMsg := core.Message{
+	messages := append([]core.Message{s.systemPromptMessage()}, s.GetMessages()...)
+	return s.llmClient.StreamChat(ctx, messages, s.EffectiveToolRegistry(), opts...)
+}
+
+func (s *AppState) systemPromptMessage() core.Message {
+	return core.Message{
 		Role:    core.RoleSystem,
 		Content: llm.Build(s.workingDir, s.SkillsCatalog(), s.SubagentsCatalog(), s.mode),
 	}
-	messages := append([]core.Message{systemMsg}, s.GetMessages()...)
-	return s.llmClient.StreamChat(ctx, messages, s.EffectiveToolRegistry(), opts...)
 }
 
 func (s *AppState) buildCompactionRequest(cfg *config.ResolvedConfig, extraPrompt string) ([]core.Message, error) {
@@ -201,24 +202,24 @@ func (s *AppState) buildCompactionRequest(cfg *config.ResolvedConfig, extraPromp
 
 	snapshot := s.GetMessages()
 	requestMessages := make([]core.Message, 0, len(snapshot)+2)
-	requestMessages = append(requestMessages, core.Message{
-		Role:    core.RoleSystem,
-		Content: llm.BuildCompactionPrompt(extraPrompt),
-	})
+	requestMessages = append(requestMessages, s.systemPromptMessage())
 	requestMessages = append(requestMessages, snapshot...)
 	requestMessages = append(requestMessages, core.Message{
 		Role:    core.RoleUser,
-		Content: compactionUserInstruction,
+		Content: llm.BuildCompactionPrompt(extraPrompt),
 	})
 	return requestMessages, nil
 }
 
-func (s *AppState) StreamCompact(ctx context.Context, cfg *config.ResolvedConfig, extraPrompt string, opts ...core.StreamOptions) (<-chan core.StreamEvent, error) {
+func (s *AppState) StreamCompact(ctx context.Context, cfg *config.ResolvedConfig, extraPrompt string, opts core.StreamOptions) (<-chan core.StreamEvent, error) {
 	requestMessages, err := s.buildCompactionRequest(cfg, extraPrompt)
 	if err != nil || requestMessages == nil {
 		return nil, err
 	}
-	return s.llmClient.StreamChat(ctx, requestMessages, nil, opts...)
+	// Keep the request prefix identical to a regular turn so provider prompt caches stay warm.
+	opts.DisableToolCalls = true
+	opts.DisableAutoCompaction = true
+	return s.llmClient.StreamChat(ctx, requestMessages, s.EffectiveToolRegistry(), opts)
 }
 
 func (s *AppState) StreamBtw(ctx context.Context, question string, opts ...core.StreamOptions) (<-chan core.StreamEvent, error) {

internal/cli/repl/appstate/state_test.go

@@ -16,7 +16,7 @@ import (
 )
 
 type mockLLMClient struct {
-	streamChatFunc func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry) (<-chan core.StreamEvent, error)
+	streamChatFunc func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry, opts []core.StreamOptions) (<-chan core.StreamEvent, error)
 	resetCount     int
 }
 
@@ -34,7 +34,7 @@ func (d dummyTool) Execute(ctx context.Context, input any) (any, error) { return
 
 func (m *mockLLMClient) StreamChat(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry, opts ...core.StreamOptions) (<-chan core.StreamEvent, error) {
 	if m.streamChatFunc != nil {
-		return m.streamChatFunc(ctx, messages, toolRegistry)
+		return m.streamChatFunc(ctx, messages, toolRegistry, opts)
 	}
 	ch := make(chan core.StreamEvent)
 	close(ch)
@@ -241,7 +241,7 @@ func TestAppState_StreamChat_WithClient(t *testing.T) {
 	var capturedMessages []core.Message
 
 	client := &mockLLMClient{
-		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry) (<-chan core.StreamEvent, error) {
+		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry, _ []core.StreamOptions) (<-chan core.StreamEvent, error) {
 			capturedMessages = append([]core.Message(nil), messages...)
 			ch := make(chan core.StreamEvent)
 			go func() {
@@ -308,7 +308,7 @@ func TestAppState_StreamChatPlanModeUsesPlanPromptAndRemovesWriteTools(t *testin
 	var capturedMessages []core.Message
 	var capturedRegistry *tools.Registry
 	client := &mockLLMClient{
-		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry) (<-chan core.StreamEvent, error) {
+		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry, _ []core.StreamOptions) (<-chan core.StreamEvent, error) {
 			capturedMessages = append([]core.Message(nil), messages...)
 			capturedRegistry = toolRegistry
 			ch := make(chan core.StreamEvent)
@@ -362,7 +362,7 @@ func TestAppState_StreamChatPlanModeUsesPlanPromptAndRemovesWriteTools(t *testin
 func TestAppState_StreamChat_ClientError(t *testing.T) {
 	expectedErr := errors.New("stream error")
 	client := &mockLLMClient{
-		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry) (<-chan core.StreamEvent, error) {
+		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry, _ []core.StreamOptions) (<-chan core.StreamEvent, error) {
 			return nil, expectedErr
 		},
 	}
@@ -458,11 +458,13 @@ func TestAppState_UpdateClient_ToNil(t *testing.T) {
 func TestAppState_StreamCompactBuildsCompactionRequest(t *testing.T) {
 	var capturedMessages []core.Message
 	var capturedRegistry *tools.Registry
+	var capturedOpts []core.StreamOptions
 
 	client := &mockLLMClient{
-		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry) (<-chan core.StreamEvent, error) {
+		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry, opts []core.StreamOptions) (<-chan core.StreamEvent, error) {
 			capturedMessages = append([]core.Message(nil), messages...)
 			capturedRegistry = toolRegistry
+			capturedOpts = append([]core.StreamOptions(nil), opts...)
 
 			ch := make(chan core.StreamEvent, 2)
 			ch <- core.StreamEvent{Type: core.StreamEventTypeChunk, Content: "compacted summary"}
@@ -487,25 +489,29 @@ func TestAppState_StreamCompactBuildsCompactionRequest(t *testing.T) {
 	eventCh, err := state.StreamCompact(context.Background(), &config.ResolvedConfig{
 		APIKey: "key",
 		Model:  "model",
-	}, "Keep business logic details")
+	}, "Keep business logic details", core.StreamOptions{SessionID: "compact-session"})
 	if err != nil {
 		t.Fatalf("StreamCompact() returned error: %v", err)
 	}
 	if eventCh == nil {
 		t.Fatal("expected compaction stream")
 	}
 
-	if capturedRegistry != nil {
-		t.Fatal("expected compaction to disable tools")
+	if capturedRegistry != state.EffectiveToolRegistry() {
+		t.Fatal("expected compaction to reuse the normal tool registry for prompt-cache parity")
+	}
+	if len(capturedOpts) != 1 || !capturedOpts[0].DisableAutoCompaction || !capturedOpts[0].DisableToolCalls || capturedOpts[0].SessionID != "compact-session" {
+		t.Fatalf("expected compaction options to preserve the session ID, disable tool calls, and block nested automatic compaction, got %#v", capturedOpts)
 	}
 	if len(capturedMessages) != len(original)+2 {
 		t.Fatalf("expected %d compaction request messages, got %d", len(original)+2, len(capturedMessages))
 	}
 	if capturedMessages[0].Role != core.RoleSystem {
 		t.Fatalf("expected first compaction message to be system, got %s", capturedMessages[0].Role)
 	}
-	if !strings.Contains(capturedMessages[0].Content, "Keep business logic details") {
-		t.Fatalf("expected extra prompt in system prompt, got %q", capturedMessages[0].Content)
+	wantSystem := llm.Build(state.WorkingDir(), state.SkillsCatalog(), state.SubagentsCatalog(), state.Mode())
+	if capturedMessages[0].Content != wantSystem {
+		t.Fatalf("expected the normal agent system prompt, got %q", capturedMessages[0].Content)
 	}
 	for i, msg := range original {
 		got := capturedMessages[i+1]
@@ -517,7 +523,7 @@ func TestAppState_StreamCompactBuildsCompactionRequest(t *testing.T) {
 	if last.Role != core.RoleUser {
 		t.Fatalf("expected final compaction message to be user, got %s", last.Role)
 	}
-	if last.Content != compactionUserInstruction {
+	if last.Content != llm.BuildCompactionPrompt("Keep business logic details") {
 		t.Fatalf("unexpected final compaction instruction: %q", last.Content)
 	}
 }
@@ -545,7 +551,7 @@ func TestAppState_StreamBtwBuildsCorrectMessages(t *testing.T) {
 	var capturedRegistry *tools.Registry
 
 	client := &mockLLMClient{
-		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry) (<-chan core.StreamEvent, error) {
+		streamChatFunc: func(ctx context.Context, messages []core.Message, toolRegistry *tools.Registry, _ []core.StreamOptions) (<-chan core.StreamEvent, error) {
 			capturedMessages = append([]core.Message(nil), messages...)
 			capturedRegistry = toolRegistry
 
@@ -726,7 +732,7 @@ func TestAppState_StreamCompactLeavesMessagesUntouchedOnCancel(t *testing.T) {
 	eventCh, err := state.StreamCompact(ctx, &config.ResolvedConfig{
 		APIKey: "key",
 		Model:  "model",
-	}, "")
+	}, "", core.StreamOptions{})
 	if err != nil {
 		t.Fatalf("expected nil error from StreamCompact, got %v", err)
 	}

internal/cli/repl/handlers.go

@@ -4,9 +4,10 @@ import (
 	"context"
 	"errors"
 	"fmt"
-	"github.com/mochow13/keen-code/internal/llm/core"
 	"strings"
 
+	"github.com/mochow13/keen-code/internal/llm/core"
+
 	tea "charm.land/bubbletea/v2"
 	"github.com/mochow13/keen-code/internal/cli/repl/appstate"
 	replcommands "github.com/mochow13/keen-code/internal/cli/repl/commands"
@@ -98,6 +99,9 @@ func (m *replModel) handleLLMDone() (replModel, tea.Cmd) {
 func (m *replModel) handleLLMIncomplete(err error) (replModel, tea.Cmd) {
 	m.flushStreamRender()
 	m.clearAskUser()
+	if m.compaction.active && m.compaction.mode != compactionAutomatic {
+		return m.handleCompactionError(err)
+	}
 	segments := cloneStreamSegments(m.stream.handler.segments)
 	m.recordHistoricalToolActivity(segments)
 	partialResponse := m.stream.handler.GetResponse()
@@ -244,10 +248,34 @@ func (m *replModel) handleAutoCompactionStopped() (replModel, tea.Cmd) {
 	return *m, m.waitForAsyncEvent()
 }
 
+func finalAssistantRun(segments []streamSegment) string {
+	start := len(segments)
+	for start > 0 && segments[start-1].kind == segmentAssistant {
+		start--
+	}
+	var content strings.Builder
+	for _, segment := range segments[start:] {
+		content.WriteString(segment.content)
+	}
+	return content.String()
+}
+
+func hasNonTextActivity(segments []streamSegment) bool {
+	for _, segment := range segments {
+		if segment.kind != segmentAssistant && segment.kind != segmentReasoning {
+			return true
+		}
+	}
+	return false
+}
+
 func (m *replModel) handleCompactionDone() (replModel, tea.Cmd) {
 	m.flushStreamRender()
 	segments := cloneStreamSegments(m.stream.handler.segments)
 	responseLines, summary := m.stream.handler.HandleDone()
+	if hasNonTextActivity(segments) {
+		summary = finalAssistantRun(segments)
+	}
 	m.compaction.active = false
 	m.compaction.mode = compactionNone
 	m.stopLoading()