Skip to content

feat(operator): add TriageRun SDK - #104

Merged
amwarrier merged 3 commits into
mainfrom
codex/triage-run-sdk
Aug 3, 2026
Merged

feat(operator): add TriageRun SDK#104
amwarrier merged 3 commits into
mainfrom
codex/triage-run-sdk

Conversation

@amwarrier

@amwarrier amwarrier commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Summary of Changes

Jira: N/A

Adds the shared WSM SDK operations Watchtower and other consumers use to manage application diagnostic runs:

  • CreateTriageRun creates a fresh namespaced apps.wandb.com/v2 TriageRun
  • ListTriageRuns returns newest-first history, optionally filtered by application
  • GetTriageRun returns the latest phase, summary, structured results, evidence, and remediation
  • DeleteTriageRun removes terminal history only

Each creation uses metadata.generateName, so requests at T0 and T1 create distinct immutable run objects rather than updating or replaying an earlier run. Requests default to the default action and validate namespace/application identifiers before touching the cluster.

Deletion deliberately rejects Pending and Running runs. Deleting an active TriageRun would implicitly cancel its owned Job, and cancellation should be a separate explicit contract. Terminal deletion uses a Kubernetes UID precondition to avoid deleting a replacement object with the same name.

The implementation uses unstructured.Unstructured rather than coupling WSM to an unreleased operator API package. It reuses WSM's existing cached dynamic Kubernetes client.

Dependency-ordered rollout

Test Plan

  • GOWORK=off GOCACHE=/private/tmp/wsm-triage-go-cache go test ./...
  • GOWORK=off GOCACHE=/private/tmp/wsm-triage-go-cache go vet ./...
  • git diff --check

Requirements

  • Tests pass
  • Vet passes
  • Formatting is clean
  • Exported SDK contracts are documented with GoDoc

Summary by CodeRabbit

  • New Features

    • Added support for creating, listing, viewing, and deleting triage runs.
    • Triage runs now provide validated request details, status summaries, diagnostic results, and filtered, sorted history.
    • Deletion is restricted to triage runs that have reached a terminal state.
  • Tests

    • Added comprehensive coverage for validation, lifecycle operations, filtering, sorting, status parsing, metadata, and error handling.

@amwarrier amwarrier added the release:minor Release: bump minor version (vX.Y+1.0) label Jul 28, 2026 — with ChatGPT Codex Connector
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f6b8f1a6-3c4e-4cf6-9f87-dc59ee4d05b9

📥 Commits

Reviewing files that changed from the base of the PR and between cfcffc1 and 5ccd072.

📒 Files selected for processing (2)
  • pkg/operator/triage.go
  • pkg/operator/triage_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • pkg/operator/triage_test.go
  • pkg/operator/triage.go

📝 Walkthrough

Walkthrough

Adds a Kubernetes dynamic-client API for TriageRun creation, listing, retrieval, and protected deletion. The API validates inputs, applies defaults, generates metadata, parses status data, sorts history, and exposes typed results with contextual errors.

Changes

TriageRun API

Layer / File(s) Summary
Create API and resource contracts
pkg/operator/triage.go, pkg/operator/triage_test.go
Adds public TriageRun types and lifecycle entry points. Creation validates requests, applies the default action, generates bounded names, sets metadata, and tests API behavior and error handling.
History and status conversion
pkg/operator/triage.go, pkg/operator/triage_test.go
Lists and filters TriageRun resources, sorts them newest first, and converts phases, summaries, verdict counts, durations, and diagnostic evidence.
Retrieve and protected deletion
pkg/operator/triage.go, pkg/operator/triage_test.go
Retrieves typed TriageRun data. Deletion requires Succeeded or Failed and uses a UID precondition. Tests cover retrieval and terminal-phase deletion.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Caller
  participant TriageRunAPI
  participant KubernetesDynamicClient
  participant TriageRunResource
  Caller->>TriageRunAPI: create TriageRunRequest
  TriageRunAPI->>KubernetesDynamicClient: create TriageRun
  KubernetesDynamicClient->>TriageRunResource: store resource
  TriageRunResource-->>KubernetesDynamicClient: return resource reference
  KubernetesDynamicClient-->>TriageRunAPI: return namespace and name
  TriageRunAPI-->>Caller: return TriageRunRef
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description check ✅ Passed The description includes the required summary, Jira field, test plan, and requirements sections, with clear implementation details and validation commands.
Title check ✅ Passed The title clearly summarizes the primary change and follows the repository's Conventional Commits format.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/triage-run-sdk

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@amwarrier amwarrier changed the title feat(operator): add TriageRun creation SDK feat(operator): add TriageRun SDK Jul 28, 2026
@amwarrier
amwarrier marked this pull request as ready for review July 31, 2026 19:40
@amwarrier
amwarrier requested a review from a team as a code owner July 31, 2026 19:40

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
pkg/operator/triage.go (1)

176-216: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Consider server-side filtering as TriageRun history grows.

listTriageRuns fetches every TriageRun in the namespace with metav1.ListOptions{} (Line 190-193), then discards non-matching applications after parsing each item (Line 204-206). There is no label selector and no Limit/Continue pagination. As run history accumulates across applications in a shared namespace, this list-then-filter approach does more API server and client work than necessary on every call.

Add an application-identifying label at creation time (alongside triageRunManagedByLabel in newTriageRun, Lines 465-467) and filter with a label selector when ApplicationName is set. This reduces the payload size and parsing cost for namespaces with many runs across applications.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/operator/triage.go` around lines 176 - 216, Add a stable
application-identifying label in newTriageRun alongside triageRunManagedByLabel,
using the application name value. In listTriageRuns, build ListOptions with a
label selector for that label when request.ApplicationName is set, while
retaining the unfiltered listing behavior otherwise; continue validating and
applying the existing application-name filtering as needed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/operator/triage.go`:
- Around line 255-261: Update deleteTriageRun’s phase handling to default an
empty status.phase to "Pending" before the terminal-state comparison and error
message, matching parseTriageRun’s behavior. Preserve the existing Succeeded and
Failed handling and ErrTriageRunNotTerminal response.
- Around line 102-174: Update CreateTriageRun/createTriageRun to wait for
operator readiness via Operator readiness handling before attempting the
mutation, ensuring webhook and deployment dependencies are available. Replace
the direct dynamicClient Resource(...).Create call and its wsm FieldManager with
the established operator-approved v2 application/CR mutation path used for
TriageRun resources, while preserving request validation, default action
handling, and returned reference behavior.

---

Nitpick comments:
In `@pkg/operator/triage.go`:
- Around line 176-216: Add a stable application-identifying label in
newTriageRun alongside triageRunManagedByLabel, using the application name
value. In listTriageRuns, build ListOptions with a label selector for that label
when request.ApplicationName is set, while retaining the unfiltered listing
behavior otherwise; continue validating and applying the existing
application-name filtering as needed.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: e36e3eb4-1b19-464f-83ab-da0ce933910b

📥 Commits

Reviewing files that changed from the base of the PR and between 20e48f1 and cfcffc1.

📒 Files selected for processing (2)
  • pkg/operator/triage.go
  • pkg/operator/triage_test.go

Comment thread pkg/operator/triage.go
Comment on lines +102 to +174
// CreateTriageRun creates a fresh TriageRun through wsm's configured dynamic
// Kubernetes client. It always uses generateName: repeated calls represent
// distinct execution requests rather than updates or retries of an earlier run.
func CreateTriageRun(ctx context.Context, request TriageRunRequest) (TriageRunRef, error) {
_, dynamicClient, err := kubectl.GetDynamicClientset()
if err != nil {
return TriageRunRef{}, err
}
return createTriageRun(ctx, dynamicClient, request)
}

// ListTriageRuns returns newest-first run history in one namespace. Supplying
// ApplicationName filters the history without excluding runs created outside
// wsm.
func ListTriageRuns(ctx context.Context, request ListTriageRunsRequest) ([]TriageRun, error) {
_, dynamicClient, err := kubectl.GetDynamicClientset()
if err != nil {
return nil, err
}
return listTriageRuns(ctx, dynamicClient, request)
}

// GetTriageRun returns one run and its latest status.
func GetTriageRun(ctx context.Context, namespace, name string) (TriageRun, error) {
_, dynamicClient, err := kubectl.GetDynamicClientset()
if err != nil {
return TriageRun{}, err
}
return getTriageRun(ctx, dynamicClient, namespace, name)
}

// DeleteTriageRun removes a completed run. Pending and Running runs are not
// deleted because deletion would implicitly cancel their owned Job; cancellation
// needs a separate explicit contract.
func DeleteTriageRun(ctx context.Context, namespace, name string) error {
_, dynamicClient, err := kubectl.GetDynamicClientset()
if err != nil {
return err
}
return deleteTriageRun(ctx, dynamicClient, namespace, name)
}

func createTriageRun(
ctx context.Context,
dynamicClient dynamic.Interface,
request TriageRunRequest,
) (TriageRunRef, error) {
if err := validateTriageRunRequest(request); err != nil {
return TriageRunRef{}, err
}
if request.Action == "" {
request.Action = DefaultTriageAction
}

created, err := dynamicClient.Resource(triageRunsV2GVR).Namespace(request.Namespace).Create(
ctx,
newTriageRun(request),
metav1.CreateOptions{FieldManager: "wsm"},
)
if err != nil {
return TriageRunRef{}, fmt.Errorf(
"failed to create TriageRun for Application %s/%s: %w",
request.Namespace,
request.ApplicationName,
err,
)
}

return TriageRunRef{
Namespace: created.GetNamespace(),
Name: created.GetName(),
}, nil
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail
echo "== WaitForOperator usages =="
rg -n "WaitForOperator" -g '*.go' -C3
echo "== FieldManager wsm usages =="
rg -n 'FieldManager:\s*"wsm"' -g '*.go' -C3
echo "== dynamic client Apply usages in pkg/operator =="
rg -n '\.Apply\(' -g '*.go' pkg/operator
echo "== GetDynamicClientset definition =="
rg -n "func GetDynamicClientset" -g '*.go' -A 30

Repository: wandb/wsm

Length of output: 176


🏁 Script executed:

#!/bin/bash
set -u
echo "== candidate files =="
git ls-files '*.go' | sed -n '1,200p'
echo "== wait operator exact =="
rg -n "WaitForOperator|Webhook|CA bundle|webhook" -g '*.go' . -C 2 || true
echo "== list operator triage =="
git ls-files 'pkg/operator/*.go' 'pkg/helm/*.go' | sed -n '1,200p'
echo "== triage.go outline =="
ast-grep outline pkg/operator/triage.go --view expanded || true
echo "== triage.go relevant slices =="
wc -l pkg/operator/triage.go
sed -n '1,230p' pkg/operator/triage.go
echo "== dynamic clientset usages =="
rg -n "GetDynamicClientset|dynamicClient" -g '*.go' . -C 3 || true
echo "== v2 tria runs references =="
rg -n "triageRunsV2GVR|TriageRun|Server-side|patch\(|apply\(|Create\(|GenerateName|fieldManager" -g '*.go' . -C 2 || true

Repository: wandb/wsm

Length of output: 50365


🏁 Script executed:

#!/bin/bash
set -u

echo "== GetDynamicClientset =="
sed -n '120,155p' pkg/kubectl/kubectl.go

echo "== operator readiness wait implementation =="
sed -n '954,1028p' pkg/operator/operator.go

echo "== static call/condition analysis around CreateTriageRun path =="
python3 - <<'PY'
from pathlib import Path
triage = Path("pkg/operator/triage.go").read_text()
op = Path("pkg/operator/operator.go").read_text()
kubectl = Path("pkg/kubectl/kubectl.go").read_text()

checks = {
    "CreateTriageRun_directly_returns_createTriageRun": "return createTriageRun(ctx, dynamicClient, request)" in triage,
    "createTriageRun_unconditionally_calls_dynamic_client_Create": triage.count("dynamicClient.Resource(triageRunsV2GVR).Namespace(request.Namespace).Create(") >= 1,
    "createTriageRun_no_before_Create_WaitForOperator_call": triage[:triage.index("dynamicClient.Resource(triageRunsV2GVR).Namespace(request.Namespace).Create(")].count("WaitForOperator") == 0,
    "createTriageRun_uses_CreateOptions_FieldManager_wsm": "metav1.CreateOptions{FieldManager: \"wsm\"}" in triage,
    "createTriageRun_no_Patch_Apply_call": triage[:triage.index("dynamicClient.Resource(triageRunsV2GVR).Namespace(request.Namespace).Create(")].count(".Apply(") == 0 and triage[:triage.index("dynamicClient.Resource(triageRunsV2GVR).Namespace(request.Namespace).Create(")].count(".Patch(") == 0,
    "WaitForOperator_no_argument": "func WaitForOperator(ctx context.Context, namespace string, timeout time.Duration) error" in op,
    "GetDynamicClientset_no_timeout_context_argument": "func GetDynamicClientset() (*rest.Config, *dynamic.DynamicClient, error)" in kubectl,
}
for k, v in checks.items():
    print(f"{k}={v}")
PY

Repository: wandb/wsm

Length of output: 3629


Wait for operator readiness and use an operator-approved v2 CR mutation path.

CreateTriageRun calls kubectl.GetDynamicClientset(), which only initializes the Kubernetes REST config/client, then creates TriageRun with .Create(). This path does not call operator.WaitForOperator, so it can race the webhook CA bundle and operator deployment. Also, TriageRun is a v2 CR, so create it through an established v2 application/CR mutation path instead of mixing FieldManager: "wsm" into a Kubernetes CreateOptions on an operator-managed resource.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/operator/triage.go` around lines 102 - 174, Update
CreateTriageRun/createTriageRun to wait for operator readiness via Operator
readiness handling before attempting the mutation, ensuring webhook and
deployment dependencies are available. Replace the direct dynamicClient
Resource(...).Create call and its wsm FieldManager with the established
operator-approved v2 application/CR mutation path used for TriageRun resources,
while preserving request validation, default action handling, and returned
reference behavior.

Source: Coding guidelines

Comment thread pkg/operator/triage.go

@collinol collinol left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved but make this change #104 (comment)

@amwarrier
amwarrier merged commit 7fb59e5 into main Aug 3, 2026
14 checks passed
@amwarrier
amwarrier deleted the codex/triage-run-sdk branch August 3, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:minor Release: bump minor version (vX.Y+1.0)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants