fix(network): fail closed when credential placeholders cannot be rewritten

[TonyLuo-NV]

Summary

Makes the sandbox egress proxy fail closed when it cannot rewrite a credential placeholder, instead of forwarding the literal openshell:resolve:env:<NAME> token (or its provider-alias form) to the upstream. Closes two fail-open paths that leak the reserved placeholder on the wire and cause upstream auth failures when internal state is degraded.

Related Issue

Fixes #2161 — [DGX Spark] gateway never rewrites openshell:resolve:env: credential placeholder on sandbox egress — literal placeholder sent on the wire.

Full chain for reviewers:

• Root-cause issue (this repo): [DGX Spark] gateway never rewrites openshell:resolve:env: credential placeholder on sandbox egress — literal placeholder sent on the wire (works on DGX Station, same versions) #2161
• This PR: fail-closed hardening for both placeholder-leak paths
• Downstream bug report: [DGX Spark][Sandbox] managed MCP unusable at runtime — OpenShell never resolves the openshell:resolve:env: credential placeholder (sends it literally); mcp status stays all-green NemoClaw#6379 — managed MCP unusable on a DGX Spark host while mcp status stayed all-green
• Downstream diagnostic PR: fix(mcp): probe wire-level credential resolution in mcp status and add (#6379) NemoClaw#6382 — wire-level credential-resolution probe in mcp status / mcp add

The invariant enforced here: a live credential placeholder must never leave the proxy unresolved. Degraded internal state fails closed; explicit policy opt-outs (tls: skip) keep working.

Changes

Leak G3 — resolver-None fail-open (crates/openshell-core/src/secrets.rs)

rewrite_http_header_block took an early return that forwarded the header block verbatim whenever resolver was None. The existing fail-closed marker scan runs only after that early return (on the resolved path), so a header block carrying a live placeholder was passed through untouched. It now scans the header region for the reserved markers (openshell:resolve:env: prefix and the OPENSHELL-RESOLVE-ENV- alias marker, raw and percent-decoded) even when there is no resolver, and returns UnresolvedPlaceholderError { location: "header" } if one is present. The scan window mirrors the resolved-path scan (header_end + 256). Marker-free traffic still passes through unchanged, so credential-free endpoints are unaffected.

Leak G2 — tls_state == None raw-tunnel fallback (crates/openshell-supervisor-network/src/proxy.rs)

When TLS was detected on a CONNECT but tls_state was None (ephemeral CA generation or CA file write failed at startup, see run.rs), handle_tcp_connection emitted a Low-severity event and fell back to a raw copy_bidirectional tunnel — silently bypassing credential rewrite. Inside the proxy handler tls_state is None only on CA-init failure: the mode != Proxy case never starts this handler, and tls: skip is handled by an earlier early return. The handler now refuses the connection with a 503 Service Unavailable JSON error and a High-severity Denied/Blocked denial event instead of tunneling. The explicit tls: skip opt-out and all other modes are untouched.

Severity escalation (crates/openshell-supervisor-network/src/run.rs)

The two startup CA-failure logs ("Failed to write CA files…" and "Failed to generate ephemeral CA…") are raised from Medium to High, since a disabled TLS termination now fails closed on TLS-bearing connections.

Testing

All run locally with the sandbox HTTP proxy unset (the sandbox HTTP_PROXY=127.0.0.1:8118 intercepts loopback connections and makes the inference-routing socket tests fail spuriously; they pass with it unset and are unrelated to this change).

• cargo test -p openshell-core secrets:: → 58 passed, 0 failed. New unit tests: placeholder in request line / header value / provider-alias form with resolver=None all fail closed; the pre-existing marker-free resolver=None passthrough test still passes.
• cargo test -p openshell-supervisor-network → 812 lib tests + integration passed, 0 failed. The relay integration test (relay_request_without_resolver_fails_closed_on_placeholder) now asserts the request is rejected before any byte reaches upstream; a new test_json_error_response_503_tls_termination_unavailable locks the G2 fail-closed response contract.
• cargo fmt --all -- --check → clean.
• cargo clippy -p openshell-core -p openshell-supervisor-network --all-targets -- -D warnings → clean.

A true connection-level test of the G2 branch was not added: handle_tcp_connection requires a real TcpStream with a peeked TLS ClientHello and no existing test drives it, so the observable fail-closed response contract is locked via the build_json_error_response test instead.

Two pre-existing tests that documented the vulnerable behavior were updated to assert the fail-closed outcome:

• secrets.rs: no_resolver_passes_through_without_scanning → split into request-line / header-value / alias-marker fail-closed tests.

• l7/rest.rs: relay_request_without_resolver_leaks_placeholders → relay_request_without_resolver_fails_closed_on_placeholder.

• Unit tests added/updated

• E2E tests added/updated (if applicable)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)
• Architecture docs updated (if applicable)

Signed-off-by: Tony Luo xialuo@nvidia.com

🤖 Generated with Claude Code

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

All contributors have signed the DCO ✍️ ✅
_{Posted by the DCO Assistant Lite bot.}

[TonyLuo-NV]

I have read the DCO document and I hereby sign the DCO.

[johntmyers]

gator-agent

PR Review Status

Validation: this PR is project-valid because it is a concentrated security hardening fix for linked issue #2161, which documents a reproducible sandbox egress credential-placeholder leak path.
Head SHA: 1c567f94a73301139c6306d822f9f09688ebcf56

Review findings:

• crates/openshell-supervisor-network/src/proxy.rs:1039 sends HTTP/1.1 200 Connection Established before TLS detection, but the new failure branch writes the intended 503 later at crates/openshell-supervisor-network/src/proxy.rs:1247. For normal CONNECT clients, the first observable response is still success; the JSON 503 is sent inside the tunnel and will likely surface as a TLS handshake/protocol error instead of the documented tls_termination_unavailable response. It also logs an allowed CONNECT before the denial. Please make CA-init failure a pre-200 Connection Established decision for non-tls: skip CONNECTs, and add a connection-level test that asserts the first bytes returned to the client are the 503 when tls_state is unavailable.

Docs: no Fern docs update is required for this code-only degraded-state hardening.

Next state: gator:in-review

[TonyLuo-NV]

Re: @johntmyers gator-agent finding on the post-200 in-tunnel 503

Thanks — this was a real flaw. Fixed in e736ee9c. Because the peek-based TLS detection is inherently post-200 (the client only sends its ClientHello after reading the 200), the refusal had to move earlier, before the tunnel is acknowledged.

What changed (pre-200 decision):

• The CA-init-failure refusal is now a gate evaluated before HTTP/1.1 200 Connection Established, right after the OPA allow/deny decision (proxy.rs:756, the 200 for this path is later at proxy.rs:1086). The lookup used is query_tls_mode(&opa_engine, &decision, &host_lc, port), which resolves purely from the policy decision + host/port with no peeked bytes, so the route's TLS treatment (tls: skip vs. terminate) is known pre-200.
• When tls_state is None (ephemeral CA failed at startup) and the route is not tls: skip, the new helper refuse_connect_when_tls_unavailable (proxy.rs:4445) writes the 503 tls_termination_unavailable JSON as the first bytes on the socket and returns true; the handler then emits the High-severity Denied/Blocked OCSF event + denial notification and closes. A normal CONNECT client now reads the documented 503 instead of a TLS protocol error.
• tls: skip routes are exempt and tunnel raw exactly as before.
• No "allowed CONNECT" event precedes the denial — that log lives after the 200 (now unreachable for the refused path), so the refusal emits only the Denied event.

Defense-in-depth choice: the old post-200 in-tunnel refusal branch (proxy.rs:1255) is now unreachable in normal operation (the pre-200 gate handles it, and tls: skip raw-tunnels before the peek). I kept it as belt-and-suspenders in case a future refactor bypasses the gate, but it no longer writes an in-tunnel 503 (which was the reported bug). It fails closed by dropping the connection instead of raw-tunneling — dropping avoids both the credential leak and corrupting the client's already-established TLS handshake with meaningless HTTP bytes.

New connection-level tests (proxy.rs, real loopback TcpStream pair):

• connect_without_tls_termination_refused_with_503_before_200 — terminating route, tls_state unavailable: asserts the first bytes the client reads back start with HTTP/1.1 503 Service Unavailable (and do not contain 200 Connection Established), with body error = tls_termination_unavailable.
• connect_with_tls_termination_or_skip_proceeds_without_writing — asserts the gate writes nothing (client reads clean EOF) when TLS termination is present or the route is tls: skip.

Validation (run with sandbox proxy env unset):

env -u HTTP_PROXY -u HTTPS_PROXY -u http_proxy -u https_proxy cargo fmt --all -- --check          # clean
env -u HTTP_PROXY -u HTTPS_PROXY -u http_proxy -u https_proxy \
  cargo clippy -p openshell-supervisor-network --all-targets -- -D warnings                        # clean
env -u HTTP_PROXY -u HTTPS_PROXY -u http_proxy -u https_proxy cargo test -p openshell-supervisor-network
  # lib: 814 passed; 0 failed  (811 prior + 3 new)
  # integration: system_inference 3 passed, websocket_upgrade 2 passed, sigv4 5 ignored (need LocalStack)
env -u HTTP_PROXY -u HTTPS_PROXY -u http_proxy -u https_proxy cargo test -p openshell-core          # 299 passed; 0 failed

No Fern docs change (code-only degraded-state hardening), matching your docs note.

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @TonyLuo-NV. I re-evaluated latest head e736ee9cc5d0fe7ef60cd6ab6e949594f35c76a4 after your update about moving the TLS-unavailable refusal before 200 Connection Established.

What I checked: crates/openshell-supervisor-network/src/proxy.rs, crates/openshell-core/src/secrets.rs, crates/openshell-supervisor-network/src/l7/rest.rs, and crates/openshell-supervisor-network/src/run.rs, including the new pre-200 gate, the tls: skip path, the resolver-None header scan, and the new regression tests.

Disposition: partially resolved. The original post-200 in-tunnel 503 issue is addressed in the implementation shape, and no Fern docs update is required for this code-only degraded-state hardening.

Remaining items:

• crates/openshell-supervisor-network/src/proxy.rs:756 adds the pre-200 gate, but the new connection-level regression at crates/openshell-supervisor-network/src/proxy.rs:7915 calls only refuse_connect_when_tls_unavailable, not handle_tcp_connection. Please add a handler-level test that sends an actual CONNECT api.example.com:443 request through handle_tcp_connection with tls_state = None, asserts the first bytes are HTTP/1.1 503 Service Unavailable and never 200 Connection Established, and also covers a real tls: skip policy path through the handler.
• crates/openshell-supervisor-network/src/proxy.rs:756 currently runs the TLS-unavailable refusal before the SSRF and allowed_ips validation that starts at crates/openshell-supervisor-network/src/proxy.rs:793. During CA-init failure, an internal-address CONNECT that would normally produce 403 ssrf_denied and SSRF telemetry now returns 503 tls_termination_unavailable first. This is still fail-closed, but it weakens operator visibility for blocked internal probes during degraded state. Please keep computing effective_tls_skip early, but perform the TLS-unavailable refusal after SSRF validation and before HTTP/1.1 200 Connection Established.
• In crates/openshell-supervisor-network/src/l7/rest.rs:4915, the test says nothing should reach upstream but only checks that captured bytes do not contain the placeholder or secret. Please make the assertion match the invariant with forwarded.is_empty().

Next state: gator:in-review

[TonyLuo-NV]

Re: @johntmyers gator re-check (partially resolved)

All three remaining items are addressed in 0148b415.

1. Ordering — refusal now runs after SSRF/allowed_ips validation. The SSRF branches no longer connect inline; they return the validated SocketAddr list (let validated_addrs = if …, proxy.rs:774). The TLS-unavailable refusal runs after that match (proxy.rs:1035) — so an internal-address CONNECT has already been denied with 403 ssrf_denied + SSRF telemetry — but before the upstream connect (proxy.rs:1077) and before 200 Connection Established. effective_tls_skip is still resolved up front (query_tls_mode, no peeked bytes) since the gate consumes it. A new handler-level test locks this: an internal-address CONNECT with tls_state = None yields the 403, not the 503.

2. Handler-level regression through handle_tcp_connection. New tests drive a real CONNECT through the full entry point (not just the extracted helper), asserting the first bytes on the socket:

• connect_handler_refuses_terminating_route_with_503_before_200 (proxy.rs:8077) — terminating route, tls_state = None: first bytes are HTTP/1.1 503 Service Unavailable, response never contains 200 Connection Established.
• connect_handler_returns_ssrf_403_for_internal_address_not_503 (proxy.rs:8116) — internal address 127.0.0.1:443: HTTP/1.1 403 Forbidden / ssrf_denied, never 503 (proves item 1's ordering end-to-end).
• connect_handler_tls_skip_route_is_not_refused (proxy.rs:8154) — a real tls: skip policy path: the handler proceeds past the gate to the tunnel connect and writes no refusal/denial.

Approach and why it's shaped this way: evaluate_opa_tcp binds process identity via /proc, and on non-Linux it denies unconditionally, so a real handler-level test can't be fully portable. These tests use the identity pattern already proven by the resolve_process_identity_..._hot_swap regression — a child /bin/bash copy opens the connection via /dev/tcp so /proc resolves it against a permissive glob policy — and are gated to Linux at runtime (CI runners are Linux; they no-op with a skip message on other platforms, so they still compile there). The 503 and 403 cases short-circuit before the upstream connect, so they never egress (the terminating target is RFC 5737 TEST-NET-3, which passes SSRF but is never actually dialed). To lock the OPA precondition where /proc isn't available, a companion test handler_test_policy_allows_glob_binary_and_reads_tls_mode (proxy.rs:8191) runs on every platform and verifies the policy shape (glob binary → Allow with matched policy; tls: skip → TlsMode::Skip; terminating → TlsMode::Auto). I did not add a bypass flag to production; handle_tcp_connection is unchanged apart from the ordering fix.

Note: I develop on macOS, so the three Linux-gated handler tests execute on CI, not locally. The policy-shape test and the connection-level helper tests below run and pass locally.

3. rest.rs fail-closed assertion. relay_request_without_resolver_fails_closed_on_placeholder now asserts forwarded.is_empty() (verified it holds) instead of merely checking the placeholder/secret are absent — the stronger invariant the test name claims.

Validation (sandbox proxy env unset):

env -u HTTP_PROXY -u HTTPS_PROXY -u http_proxy -u https_proxy cargo fmt --all -- --check          # clean
env -u HTTP_PROXY -u HTTPS_PROXY -u http_proxy -u https_proxy \
  cargo clippy -p openshell-supervisor-network --all-targets -- -D warnings                        # clean
env -u HTTP_PROXY -u HTTPS_PROXY -u http_proxy -u https_proxy cargo test -p openshell-supervisor-network
  # lib: 818 passed; 0 failed  (+4: three Linux-gated handler tests + the portable policy-shape test)
  # integration: system_inference 3, websocket_upgrade 2 passed; sigv4 5 ignored (need LocalStack)
env -u HTTP_PROXY -u HTTPS_PROXY -u http_proxy -u https_proxy cargo test -p openshell-core          # 299 passed; 0 failed

No Fern docs change (code-only degraded-state hardening), matching your note.

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @TonyLuo-NV. I re-evaluated latest head 0148b4154178870124ca10dde9245aad777fe0f2 after your update that addressed the remaining handler-level CONNECT coverage, SSRF-before-TLS-unavailable ordering, and the stricter REST relay invariant.

What I checked: crates/openshell-supervisor-network/src/proxy.rs, crates/openshell-supervisor-network/src/l7/rest.rs, crates/openshell-core/src/secrets.rs, and crates/openshell-supervisor-network/src/run.rs, including the full handle_tcp_connection gate ordering, the tls: skip path, SSRF telemetry preservation, resolver-None fail-closed scanning, and the new regression tests. I also ran the required independent code-only reviewer for this head SHA.

Disposition: resolved. The previous gator findings are addressed, and the independent reviewer found no blocking correctness, security, regression, or maintainability findings remaining.

Remaining items:

• No blocking review items remain.

Docs: no Fern docs update is required for this code-only degraded-state hardening.

E2E: I applied test:e2e because this changes network proxy behavior and provider credential fail-closed handling.

Next state: gator:watch-pipeline

[github-actions]

Label test:e2e applied for 0148b41. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[TonyLuo-NV]

CI failure on the two handler tests root-caused and fixed in f694fc0 — it was a test-client bug, not a product bug:

The test's bash client forked cat to read the proxy reply, so the client socket fd was inherited by a second process with a different binary. The identity resolver correctly denies that as ambiguous shared-socket ownership (the same invariant resolve_process_identity_denies_fork_exec_shared_socket_ambiguity pins), so the handler took the deny path instead of reaching the refusal gate. On busy CI runners the deny path's /proc fallback scan exceeded the test budget, which surfaced as "hang"; the underlying decision was the same deny.

Fix: the client script now uses only bash builtins (exec 3<>/dev/tcp/..., printf, IFS= read -r -d '') so exactly one process ever owns the socket, and the child exits on EOF instead of being killed mid-read.

Verified in a Linux (aarch64) container with rust:1.95.0:

• cargo test -p openshell-supervisor-network --lib connect_handler_ → 3 passed / 0 failed (503-before-200, SSRF-403-not-503, tls-skip-not-refused)
• full cargo test -p openshell-supervisor-network --lib → 839 passed / 0 failed
• cargo fmt --all -- --check and cargo clippy -p openshell-supervisor-network --all-targets -- -D warnings → clean

The product-side ordering fix from the previous commit is unchanged.

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @TonyLuo-NV. I re-evaluated latest head f694fc00b3ed116a9a513242ccf29067275bd827 after your update that the handler test client was made fork-free.

What I checked: crates/openshell-supervisor-network/src/proxy.rs, crates/openshell-supervisor-network/src/procfs.rs, crates/openshell-core/src/secrets.rs, and the current Branch Checks result. I also ran the required independent code-only reviewer for this head SHA.

Disposition: not resolved. The product-side fail-closed ordering still looks correct, but the current head's OpenShell / Branch Checks run still fails on Rust (linux-arm64-cpu8) in the two new handler tests.

Remaining items:

• crates/openshell-supervisor-network/src/proxy.rs:8048 sets the handler test entrypoint_pid to the test harness PID. That can miss the spawned bash client in the descendant fast path and fall back to scanning all of /proc in crates/openshell-supervisor-network/src/procfs.rs:365, which matches the current CI timeouts at proxy.rs:8103 and proxy.rs:8140. Please use the spawned bash client's PID as the entrypoint_pid in drive_connect_through_handler, so the identity resolver stays on the intended descendant path.
• crates/openshell-core/src/secrets.rs:1906 covers raw no-resolver placeholder markers, but not the percent-decoded marker path claimed by the PR. Please add a no-resolver regression with a %3A-encoded openshell:resolve:env:KEY marker in the request target or header.

Docs: no Fern docs update is required for this code-only degraded-state hardening.

E2E: test:e2e remains appropriate because this changes network proxy behavior and provider credential fail-closed handling. E2E is still pending for this head.

Next state: gator:in-review

[TonyLuo-NV]

The remaining handler-test flake is fixed in cc8aaf4 by removing the child process entirely:

Why the previous two attempts still failed on CI: any child-process client (even fork-free bash) makes the test depend on resolving the child's socket ownership through /proc. On CI runners with a busy or permission-restricted /proc, that resolution degrades into the whole-/proc fallback scan and a deny — which surfaces as the "hang" panic. The same tests pass in a quiet container, which is why it was invisible locally.

Redesign: the client is now an in-process TcpStream and the test policy allows current_exe(), so identity resolution binds the socket to the test process itself via the descendant scan — the same in-process pattern the consistently-passing resolve_process_identity_* tests rely on, with no environment-sensitive fallback path.

Non-vacuous tls: skip: the harness now threads a denial_tx and the tls: skip test asserts the handler emitted no DenialEvent at any stage, so it can no longer pass vacuously on a policy or identity deny.

Verified in a Linux (aarch64) rust:1.95.0 container, both with an isolated pid namespace and with --pid=host (busy-/proc simulation): the 3 handler tests pass in both, and the full cargo test -p openshell-supervisor-network --lib suite is 839 passed / 0 failed. cargo fmt --check and clippy -D warnings clean.

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @TonyLuo-NV. I re-evaluated latest head cc8aaf42397adf4e02f9dd6ef6dee74a9f34f02d after your update that the CONNECT handler tests now use an in-process TcpStream/current_exe() client and assert the tls: skip path emits no denial event.

What I checked: crates/openshell-supervisor-network/src/proxy.rs, crates/openshell-supervisor-network/src/l7/rest.rs, crates/openshell-core/src/secrets.rs, and the current Branch Checks/E2E status. I also ran the required independent code-only reviewer for this head SHA.

Disposition: partially resolved. The product-side fail-closed path looks correct now: SSRF/allowed_ips validation happens before the TLS-unavailable 503 gate, 200 Connection Established stays after that gate, and the tls: skip path remains exempt. The REST relay invariant also now checks that no bytes reach upstream. No Fern docs update is required for this code-only degraded-state hardening.

Remaining items:

• crates/openshell-core/src/secrets.rs:1906 still lacks the requested no-resolver regression for a percent-encoded canonical marker. The implementation should catch this through contains_reserved_credential_marker, but please add a test such as GET /api/openshell%3Aresolve%3Aenv%3AKEY HTTP/1.1 with resolver = None and assert it fails closed with UnresolvedPlaceholderError { location: "header" }.

E2E: test:e2e remains appropriate because this changes network proxy behavior and provider credential fail-closed handling. E2E is still pending for this head.

Next state: gator:in-review

[TonyLuo-NV]

@johntmyers The remaining item is addressed in cb5bc51: added no_resolver_with_percent_encoded_marker_fails_closed — GET /api/openshell%3Aresolve%3Aenv%3AKEY HTTP/1.1 with resolver = None fails closed with UnresolvedPlaceholderError { location: "header" } via the percent-decoded pass of contains_reserved_credential_marker, exactly as the implementation intends. cargo test -p openshell-core secrets:: → 59 passed / 0 failed (Linux container, rust:1.95.0); cargo fmt --check clean.

E2E: understood that test:e2e stays applied — the label-triggered run needs a maintainer Re-run all jobs on the existing workflow per the bot instructions whenever you're ready.

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @TonyLuo-NV. I re-evaluated latest head cb5bc5156bd7518f4930bb5612b1f97075f49a8b after your update that added the percent-encoded no-resolver placeholder regression.

What I checked: crates/openshell-core/src/secrets.rs, crates/openshell-supervisor-network/src/proxy.rs, crates/openshell-supervisor-network/src/l7/rest.rs, and crates/openshell-supervisor-network/src/run.rs, including the no-resolver raw and percent-decoded marker scan, the pre-200 TLS-unavailable CONNECT refusal, SSRF-before-503 ordering, the tls: skip path, and the updated regression tests. I also ran the required independent code-only reviewer for this head SHA.

Disposition: resolved. The previous gator finding is addressed, and the independent reviewer found no blocking correctness, security, regression, maintainability, or missing-test issues remaining.

Remaining items:

• No blocking review items remain.

Docs: no Fern docs update is required for this code-only degraded-state hardening.

E2E: test:e2e remains appropriate because this changes network proxy behavior and provider credential fail-closed handling. Branch Checks and Helm Lint are green; OpenShell / E2E is still pending.

Next state: gator:watch-pipeline

[TonyLuo-NV]

E2E status on cb5bc51: Branch Checks are green (the redesigned in-process handler tests passed on both Linux runners), and the only failure is e2e / E2E (rust-podman-rootless, ubuntu-26.04) on upload_respects_gitignore_by_default (tests/sync.rs) — sandbox download fails with ssh tar create exited with status 1, i.e. the file-sync path over SSH.

That surface is disjoint from this PR: the diff touches the HTTP CONNECT proxy path (SSRF→TLS-gate ordering) and the credential-placeholder header rewrite, neither of which is on the sandbox upload/download SSH tar path, and the fail-closed branches only trigger on CA-init failure or reserved markers — neither occurs in that test. The other three E2E matrix legs (docker, podman root, GPU) passed.

@johntmyers could you re-run the failed E2E job when you get a chance? (Contributors can't re-run workflows on this repo.)

[TonyLuo-NV]

Additional evidence that the rust-podman-rootless sync failure is a flake: this same PR's Branch E2E Checks ran green twice earlier today — run 29072053859 (05:44Z) and run 29067340520 (03:40Z) — on head 0148b41, whose product code is identical to the current head (the only commits since, cc8aaf4 and cb5bc51, are test-only changes to proxy.rs's test module and one secrets.rs unit test). The failing upload_respects_gitignore_by_default sync test passed in both of those runs on the same code path. A re-run of the failed job should clear it.