Skip to content

test(webview): e2e tests + API task controls (branch 3/3) #1140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
easonLiangWorldedtech wants to merge 18 commits into
base: main
Choose a base branch
from
Draft

test(webview): e2e tests + API task controls (branch 3/3) #1140

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
ee0cd01
feat(webview): add view-local state base
easonliang28 Jul 21, 2026
62989b7
fix(webview): persist per-view selections through registered global s…
easonliang28 Jul 21, 2026
1477389
fix(webview): route mode switches through view-local persistence
easonliang28 Jul 21, 2026
c24b7ba
chore: remove invisible chars
easonliang28 Jul 21, 2026
a59e01d
test(webview): restore ClineProvider parallel mode coverage
easonliang28 Jul 21, 2026
2dab0de
fix(webview): sync view local state after profile mutations
easonliang28 Jul 23, 2026
1becf87
fix(webview): persist view-local state safely
easonliang28 Jul 23, 2026
42ffba7
fix(provider): sync view-local state when activating provider profile
easonliang28 Jul 21, 2026
8727a47
fix(api): sync setConfiguration view-local state
easonliang28 Jul 24, 2026
d06d70f
test(strengthen): assert stale openRouterModelId absent after profile…
easonliang28 Jul 24, 2026
797c819
fix(webview): preserve isolated view state writes
easonliang28 Jul 27, 2026
c77de69
chore(lint): update eslint suppression baseline
easonliang28 Jul 27, 2026
7d07e53
refactor(webview): consolidate view-local state persistence
easonliang28 Jul 29, 2026
cf8fd95
test(webview): mock workspace tracker launch init
easonliang28 Jul 29, 2026
ef0f0d3
test(vscode-e2e): cover cross-panel view state isolation
easonliang28 Jul 30, 2026
601e6bc
test(vscode-e2e): cover follow-up mode isolation
easonliang28 Jul 30, 2026
e55f596
test(api): cover task controls and view-local values
easonliang28 Jul 30, 2026
7fe9f6d
fix(webview): address view-state review feedback
easonliang28 Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions apps/vscode-e2e/fixtures/modes.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,20 @@
}
]
}
},
{
"match": {
"userMessage": "Use the `switch_mode` tool to switch to debug mode."
},
"response": {
"toolCalls": [
{
"name": "switch_mode",
"arguments": "{\"mode_slug\":\"debug\",\"reason\":\"User requested to switch to debug mode.\"}",
"id": "call_modes_switch_002"
}
]
}
Comment on lines +17 to +29

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Files:"
git ls-files | rg '(^|/)modes\.json$|apps/vscode-e2e/src/runTest\.ts$|modes|debug|call_modes_switch' || true

echo
echo "Modes fixture:"
if [ -f apps/vscode-e2e/fixtures/modes.json ]; then
  cat -n apps/vscode-e2e/fixtures/modes.json
fi

echo
echo "runTest references to match/sequenceIndex/fixture tools:"
if [ -f apps/vscode-e2e/src/runTest.ts ]; then
  rg -n "sequenceIndex|match|toolCalls|fixture|call_modes_switch|modes" apps/vscode-e2e/src/runTest.ts -C 3
fi

echo
echo "Other fixtures with sequenceIndex:"
rg -n '"sequenceIndex"\s*:\s*0|sequenceIndex' apps/vscode-e2e/fixtures -S -C 2 || true

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 10770

Mark the first debug tool response as turn zero.

The JSON fixture does not include sequenceIndex, while later fixtures consume call_modes_switch_002. Add "sequenceIndex": 0 to the match object for this turn so replay associates the follow-up response with the same request sequence.

Proposed fix 
 "match": {
-  "userMessage": "Use the `switch_mode` tool to switch to debug mode."
+  "userMessage": "Use the `switch_mode` tool to switch to debug mode.",
+  "sequenceIndex": 0
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
{
"match": {
"userMessage": "Use the `switch_mode` tool to switch to debug mode."
},
"response": {
"toolCalls": [
{
"name": "switch_mode",
"arguments": "{\"mode_slug\":\"debug\",\"reason\":\"User requested to switch to debug mode.\"}",
"id": "call_modes_switch_002"
}
]
}
{
"match": {
"userMessage": "Use the `switch_mode` tool to switch to debug mode.",
"sequenceIndex": 0
},
"response": {
"toolCalls": [
{
"name": "switch_mode",
"arguments": "{\"mode_slug\":\"debug\",\"reason\":\"User requested to switch to debug mode.\"}",
"id": "call_modes_switch_002"
}
]
}
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/vscode-e2e/fixtures/modes.json` around lines 17 - 29, Update the match
object for the first debug-mode switch_mode response in modes.json to include
sequenceIndex set to 0, preserving the existing userMessage matcher and response
so replay associates the subsequent call_modes_switch_002 fixture with the
correct request sequence.

Source: Coding guidelines

}
]
}
95 changes: 95 additions & 0 deletions apps/vscode-e2e/src/fixtures/view-state.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
import type { ChatCompletionRequest, ChatMessage, LLMock } from "@copilotkit/aimock"

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🔴 Critical | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

file="apps/vscode-e2e/src/fixtures/view-state.ts"

if rg -q 'import type .*LLMock' "$file" && rg -qF 'InstanceType<typeof LLMock>' "$file"; then
	echo "LLMock is type-only imported and used in a typeof type query."
	exit 1
fi

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 221

🏁 Script executed:

#!/bin/bash
set -euo pipefail

file="apps/vscode-e2e/src/fixtures/view-state.ts"

printf 'Relevant imports and function signature:\n'
sed -n '1,12p;45,58p' "$file"

printf '\nTypeScript tooling / config evidence:\n'
for f in package.json tsconfig.json apps/vscode-e2e/tsconfig.json; do
  [ -f "$f" ] && { echo "--- $f"; sed -n '1,220p' "$f"; }
done

printf '\nAttempt type-only import usage probe without building repository code:\n'
tmpdir="$(mktemp -d)"
cat > "$tmpdir/main.ts" <<'TS'
import type { LLMock } from './LLMOCK'
export function addViewStateFixtures(mock: InstanceType<typeof LLMock>) {
  return mock
}
TS
cat > "$tmpdir/LLMOCK.ts" <<'TS'
export class LLMock {
  constructor(message: string) {}
}
TS
cat > "$tmpdir/tsconfig.json" <<'JSON'
{
  "compilerOptions": {
    "module": "esnext",
    "target": "esnext",
    "strict": true,
    "noEmit": true
  }
}
JSON
if command -v npx >/dev/null 2>&1; then
  cd "$tmpdir" && npx -y --no-install typescript@latest --project tsconfig.json
fi

Repository: Zoo-Code-Org/Zoo-Code

Length of output: 4968

Use LLMock directly as the instance type.

LLMock is imported as a type but addViewStateFixtures uses InstanceType<typeof LLMock>. A typeof type query needs a value binding, and this type-only import also violates the app’s type-check setup. Use LLMock directly for the parameter type.

Proposed fix 
-export function addViewStateFixtures(mock: InstanceType<typeof LLMock>) {
+export function addViewStateFixtures(mock: LLMock) {
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/vscode-e2e/src/fixtures/view-state.ts` at line 1, Update the
addViewStateFixtures parameter type to use the imported LLMock type directly
instead of InstanceType<typeof LLMock>; keep the existing ChatCompletionRequest
and ChatMessage typing unchanged.


const TASKS = ["A", "B", "C"] as const
const ROUNDS = 10

const MODE_SEQUENCES: Record<(typeof TASKS)[number], string[]> = {
A: ["ask", "debug", "architect", "orchestrator", "code", "ask", "debug", "architect", "orchestrator", "code"],
B: ["debug", "architect", "orchestrator", "code", "ask", "debug", "architect", "orchestrator", "code", "ask"],
C: ["architect", "orchestrator", "code", "ask", "debug", "architect", "orchestrator", "code", "ask", "debug"],
}

const markerFor = (taskName: (typeof TASKS)[number]) => `FOLLOWUP_MODE_ISOLATION_${taskName}`
const answerFor = (taskName: (typeof TASKS)[number], round: number) => `${taskName} follow-up round ${round}`
const callIdFor = (taskName: (typeof TASKS)[number], round: number) =>
`call_followup_mode_${taskName.toLowerCase()}_${String(round).padStart(2, "0")}`

const lastToolResultContains = (req: ChatCompletionRequest, toolCallId: string, expected: string[]) => {
const messages = Array.isArray(req?.messages) ? req.messages : []
const toolMessage = messages.filter((message: ChatMessage) => message?.role === "tool").at(-1)
const content = toolMessage?.content

return (
toolMessage?.tool_call_id === toolCallId &&
typeof content === "string" &&
expected.every((text) => content.includes(text))
)
}

const followupToolCall = (taskName: (typeof TASKS)[number], round: number) => ({
name: "ask_followup_question",
arguments: JSON.stringify({
question: `Task ${taskName}: choose mode for round ${round}`,
follow_up: [
{
text: answerFor(taskName, round),
mode: MODE_SEQUENCES[taskName][round - 1],
},
],
}),
id: callIdFor(taskName, round),
})

export const getFollowupModeIsolationPlan = () =>
TASKS.map((taskName) => ({
taskName,
marker: markerFor(taskName),
rounds: MODE_SEQUENCES[taskName].map((mode, index) => ({
round: index + 1,
answer: answerFor(taskName, index + 1),
mode,
})),
}))

export function addViewStateFixtures(mock: InstanceType<typeof LLMock>) {
for (const taskName of TASKS) {
mock.addFixture({
match: {
userMessage: markerFor(taskName),
},
response: {
toolCalls: [followupToolCall(taskName, 1)],
},
})

for (let round = 1; round < ROUNDS; round++) {
mock.addFixture({
match: {
predicate: (req) =>
lastToolResultContains(req, callIdFor(taskName, round), [answerFor(taskName, round)]),
},
response: {
toolCalls: [followupToolCall(taskName, round + 1)],
},
})
}

mock.addFixture({
match: {
predicate: (req) =>
lastToolResultContains(req, callIdFor(taskName, ROUNDS), [answerFor(taskName, ROUNDS)]),
},
response: {
toolCalls: [
{
name: "attempt_completion",
arguments: JSON.stringify({
result: `Task ${taskName} completed ${ROUNDS} follow-up mode switches.`,
}),
id: `call_followup_mode_${taskName.toLowerCase()}_complete`,
},
],
},
})
}
}
33 changes: 33 additions & 0 deletions apps/vscode-e2e/src/runTest.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ import { addSearchFilesResultFixtures } from "./fixtures/search-files"
import { addSubtaskFixtures } from "./fixtures/subtasks"
import { addUseMcpToolResultFixtures } from "./fixtures/use-mcp-tool"
import { addWriteToFileResultFixtures } from "./fixtures/write-to-file"
import { toolResultContains } from "./fixtures/tool-result"
import { addViewStateFixtures } from "./fixtures/view-state"

function getCliFlagValue(flag: string) {
return process.argv.find((arg, index) => process.argv[index - 1] === flag)
Expand Down Expand Up @@ -129,6 +131,37 @@ async function main() {
addUseMcpToolResultFixtures(mock)
addWriteToFileResultFixtures(mock)
addDeepSeekV4Fixtures(mock)
addViewStateFixtures(mock)

mock.addFixture({
match: {
predicate: (req) => toolResultContains(req, "call_modes_switch_001", []),
},
response: {
toolCalls: [
{
name: "attempt_completion",
arguments: JSON.stringify({ result: "Switched to ❓ Ask mode as requested." }),
id: "call_modes_post_switch_001",
},
],
},
})

mock.addFixture({
match: {
predicate: (req) => toolResultContains(req, "call_modes_switch_002", []),
},
response: {
toolCalls: [
{
name: "attempt_completion",
arguments: JSON.stringify({ result: "Switched to 🪲 Debug mode as requested." }),
id: "call_modes_post_switch_002",
},
],
},
})

// The modes test (switch_mode → ask) triggers a second API call whose last
// user message starts with <environment_details> directly — no <user_message>
Expand Down
Loading
Loading