Skip to content

[VS Code] Validate blocking E2E coverage for core developer workflows #22156

Open

Description

@adamint

Goal

Verify that blocking VS Code extension E2E coverage protects the core developer workflows tracked by #19405. Keep the evidence and any concrete coverage gaps here rather than repeating exit criteria in the parent tracker.

Existing E2E tests are the baseline. This is not a claim that those tests are missing or failing; the complete workflow-to-blocking-lane mapping and current execution evidence have not been established by the tracker refresh.

Exit criteria

  • Map onboarding, AppHost discovery, launch, resource actions, dashboard access, and teardown to their existing real-workflow E2E scenarios and CI jobs. Identify concrete gaps rather than treating test-file presence as proof of coverage.
  • Verify those scenarios run in a blocking PR lane for relevant changes, rather than only on a schedule or in a non-blocking job. Follow the repository's conditional test-selection rules; do not require unrelated PRs to run every E2E test.
  • For uncovered workflows, add or extend the existing fixtures and record the linked fix. Do not duplicate scenarios already covered.
  • Record passing execution evidence for the mapped scenarios and make remaining failures explicit. A closed implementation issue or synthetic configuration test is not a substitute for execution evidence.
  • Preserve enough diagnostic evidence to distinguish extension regressions, external debugger failures, and infrastructure/setup failures. Link concrete failures to their owning issue instead of reporting them as successful validation.

Scope

Reuse the existing VS Code extension E2E suite and CI lanes.

Per-language debugger coverage remains #18957 under #19562. Remote execution and qualification remain #19406. New feature-specific tests stay with their feature issues. Reference those results here rather than duplicating their requirements or claiming qualification across untested languages, hosts, or platforms.

User-facing actionable error handling remains #19504. Ongoing CI health and flaky-test remediation remain #21600.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions