feat: initialize E2E testing framework (Webwright + Playwright)#91
feat: initialize E2E testing framework (Webwright + Playwright)#91MrOrz wants to merge 2 commits into
Conversation
Adds tests/e2e/ scaffold for the hybrid E2E strategy: - auth/ for locally-captured OAuth state (git-ignored) - generated_scripts/ for Webwright-crafted Playwright tests accumulated over time - templates/task_template.json with cofacts/ai-specific routing & streaming hints - README.md documenting the full workflow - .gitignore rule to prevent auth_state.json from being committed - package.json scripts: test:e2e:login and test:e2e:run-all https://claude.ai/code/session_01RQBLgYKW6LqFzKXdJqfpbb
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a hybrid E2E testing framework combining Webwright and Playwright, including configuration, documentation, a task template, and initial test scripts. The feedback focuses on improving the robustness and maintainability of the setup: using npx and uv run in package.json scripts to ensure correct execution environments, deduplicating the page fixture into a shared conftest.py while gracefully handling missing authentication states, and addressing a race condition in the chat stream test by waiting for the loading class to disappear.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| "typegen": "openapi-typescript http://localhost:8000/openapi.json -o src/lib/adk-types.d.ts && sed -i '' 's/: unknown/: {}/g' src/lib/adk-types.d.ts", | ||
| "codegen": "graphql-codegen" | ||
| "codegen": "graphql-codegen", | ||
| "test:e2e:login": "playwright open --save-storage=tests/e2e/auth/auth_state.json http://localhost:3000/login", |
There was a problem hiding this comment.
The playwright package is not listed in devDependencies or dependencies in package.json. Running pnpm test:e2e:login will fail unless Playwright is installed globally, which is not recommended. Additionally, using npx ensures that the locally installed version of Playwright is used consistently.
| "test:e2e:login": "playwright open --save-storage=tests/e2e/auth/auth_state.json http://localhost:3000/login", | |
| "test:e2e:login": "npx playwright open --save-storage=tests/e2e/auth/auth_state.json http://localhost:3000/login", |
| # Wait for the ADK structured response to finish rendering | ||
| # The assistant message container should appear and stop showing a loading state | ||
| page.wait_for_selector("[data-testid='assistant-message']", timeout=RESPONSE_TIMEOUT) |
There was a problem hiding this comment.
The page.wait_for_selector call only waits for the [data-testid='assistant-message'] element to appear in the DOM. If the element appears immediately with a loading spinner or class, the test will proceed immediately before the streaming response actually finishes. This creates a race condition and potential false positives.\n\nTo ensure the streaming response has fully finished rendering, wait for the loading state/class to disappear as documented in the README.
| # Wait for the ADK structured response to finish rendering | |
| # The assistant message container should appear and stop showing a loading state | |
| page.wait_for_selector("[data-testid='assistant-message']", timeout=RESPONSE_TIMEOUT) | |
| # Wait for the ADK structured response to finish rendering | |
| # The assistant message container should appear and stop showing a loading state | |
| page.wait_for_selector("[data-testid='assistant-message']", timeout=RESPONSE_TIMEOUT) | |
| page.wait_for_selector("[data-testid='assistant-message']:not(.loading)", timeout=RESPONSE_TIMEOUT) |
| "codegen": "graphql-codegen" | ||
| "codegen": "graphql-codegen", | ||
| "test:e2e:login": "playwright open --save-storage=tests/e2e/auth/auth_state.json http://localhost:3000/login", | ||
| "test:e2e:run-all": "pytest tests/e2e/generated_scripts/" |
There was a problem hiding this comment.
Running bare pytest relies on the user having their Python virtual environment manually activated in their shell. Since the project uses uv for dependency management, running pytest via uv run ensures that the tests are executed within the correct virtual environment with all required dependencies automatically resolved.
| "test:e2e:run-all": "pytest tests/e2e/generated_scripts/" | |
| "test:e2e:run-all": "uv run pytest tests/e2e/generated_scripts/" |
| @pytest.fixture(scope="module") | ||
| def page(): | ||
| with sync_playwright() as p: | ||
| browser = p.chromium.launch() | ||
| context = browser.new_context(storage_state=AUTH_STATE) | ||
| pg = context.new_page() | ||
| yield pg | ||
| browser.close() |
There was a problem hiding this comment.
The page fixture is duplicated across multiple test files (e.g., test_chat_stream.py and test_user_profile.py). Additionally, if auth_state.json is missing (which is common in CI/CD or fresh checkouts), the fixture will crash with a FileNotFoundError.\n\nTo resolve both issues, move this fixture to a shared tests/e2e/conftest.py file and add a check to gracefully skip tests if the authentication state is missing.\n\n### Suggested tests/e2e/conftest.py:\npython\nimport os\nimport pytest\nfrom playwright.sync_api import sync_playwright\n\nAUTH_STATE = "tests/e2e/auth/auth_state.json"\n\n@pytest.fixture(scope="module")\ndef page():\n if not os.path.exists(AUTH_STATE):\n pytest.skip(f"Auth state file not found at {AUTH_STATE}. Run 'pnpm test:e2e:login' first.")\n with sync_playwright() as p:\n browser = p.chromium.launch()\n context = browser.new_context(storage_state=AUTH_STATE)\n pg = context.new_page()\n yield pg\n browser.close()\n
| @pytest.fixture(scope="module") | |
| def page(): | |
| with sync_playwright() as p: | |
| browser = p.chromium.launch() | |
| context = browser.new_context(storage_state=AUTH_STATE) | |
| pg = context.new_page() | |
| yield pg | |
| browser.close() | |
| # Fixture moved to shared conftest.py |
| @pytest.fixture(scope="module") | ||
| def page(): | ||
| with sync_playwright() as p: | ||
| browser = p.chromium.launch() | ||
| context = browser.new_context(storage_state=AUTH_STATE) | ||
| pg = context.new_page() | ||
| yield pg | ||
| browser.close() |
There was a problem hiding this comment.
The page fixture is duplicated here and should be moved to a shared tests/e2e/conftest.py file to improve maintainability and handle missing authentication states gracefully.
| @pytest.fixture(scope="module") | |
| def page(): | |
| with sync_playwright() as p: | |
| browser = p.chromium.launch() | |
| context = browser.new_context(storage_state=AUTH_STATE) | |
| pg = context.new_page() | |
| yield pg | |
| browser.close() | |
| # Fixture moved to shared conftest.py |
- Use npx playwright and uv run pytest in package.json scripts for correct execution environments without global installs - Extract shared page fixture to tests/e2e/conftest.py; gracefully skip tests when auth_state.json is missing - Fix race condition in test_chat_stream: also wait for .loading class to disappear before asserting response rendered https://claude.ai/code/session_01RQBLgYKW6LqFzKXdJqfpbb
2 participants
Summary
tests/e2e/scaffold for the hybrid Webwright + Playwright E2E strategytests/e2e/auth/holds locally-captured OAuth state (git-ignored via.gitignorerule)tests/e2e/generated_scripts/accumulates Webwright-crafted Playwright test skeletonstests/e2e/templates/task_template.jsonprovides cofacts/ai-specific routing & ADK streaming hints for writing Webwright task promptstests/e2e/README.mddocuments the full 4-step workflowpackage.jsongainstest:e2e:loginandtest:e2e:run-allscriptsTest plan
pnpm test:e2e:loginopens a browser athttp://localhost:3000/loginand saves state totests/e2e/auth/auth_state.jsontests/e2e/auth/*.jsondoes NOT appear ingit status(confirmed by.gitignorerule)pnpm test:e2e:run-allresolves topytest tests/e2e/generated_scripts/https://claude.ai/code/session_01RQBLgYKW6LqFzKXdJqfpbb
Generated by Claude Code