Skip to content

nvx-rs: document when a child joins its cgroup and what settlement counts - #922

Closed
Enrique Saurez (esaurez) wants to merge 1 commit into
devfrom
esaurez/exec-cgroup-docs-20261026
Closed

Enrique Saurez (esaurez) wants to merge 1 commit into
devfrom
esaurez/exec-cgroup-docs-20261026

Conversation

@esaurez

Copy link
Copy Markdown
Contributor

Two facts that callers of the guest library rely on hold today, but the documentation doesn't state them where callers look:

  • A child joins its cgroup in its own launch stages. With Command::cgroup, the child writes its process ID to the cgroup's cgroup.procs in its Cgroup launch stage, after it has placed its descriptors and, if it waits at a barrier, after the barrier's release (PreparedChild::run_child in nvx-rs/src/guest/sys/spawn.rs). Command::spawn returns once it has started the child, without waiting for that stage. So until the child's LaunchReport arrives, the cgroup may not hold the child, and ExecutionCgroup::is_occupied, kill, remove and settle can miss it.
  • Settlement counts zombies. The execution binding settles once ExecutionCgroup::is_occupied, which reads pids.current, finds the cgroup empty. pids.current counts a zombie until its parent collects it, while cgroup.procs and the populated entry of cgroup.events drop a process as soon as it exits. So a caller that watches those files can see the cgroup empty before the execution settles.

This PR documents both, and changes no behavior:

  • Command::cgroup: the window in which the cgroup may not hold the child, and the two ways for a caller to have the child in the cgroup first: wait for the launch report, or spawn the child with a barrier instead of the cgroup, place it with ExecutionCgroup::place, and then release the barrier.
  • ExecutionCgroup::placement: the child joins in one of its own launch stages, with a link to Command::cgroup.
  • ExecutionCgroup::is_occupied and kill: a child spawned with the placement whose launch has yet to report may not be in the cgroup yet, worded like remove's existing warning.
  • nvx::guest::exec::binding's module documentation: settlement checks ExecutionCgroup::is_occupied, which reads pids.current, and pids.current counts a zombie until its parent collects it, unlike cgroup.procs and populated.
  • A test, guest::spawn::tests::a_child_joins_its_cgroup_once_its_barrier_is_released, pins the order that the documentation now states, with the fake membership file of the existing cgroup test. A child spawned with a barrier and a cgroup has not joined while it waits at its barrier, and has joined once its launch report arrives. The test knows that the child waits once a pipe hangs up whose writer the child closes just before it waits, so it fails whenever the child joins before its barrier, however the child is scheduled.

nvx::guest::cgroup's module documentation stays as it is: #912 rewrites it with its findings.

Refs #912

Kernel references (Linux v6.18)

  • Documentation/admin-guide/cgroup-v2.rst: "After exit, a process stays associated with the cgroup that it belonged to at the time of exit until it's reaped; however, a zombie process does not appear in "cgroup.procs" [...]". A cgroup "which doesn't have any children and is associated only with zombie processes is considered empty and can be removed".
  • The same file, cgroup.events: populated is "1 if the cgroup or its descendants contains any live processes; otherwise, 0."
  • do_exit() calls cgroup_exit(), which takes the task off its css_set and updates populated, before exit_notify() makes it a zombie. cgroup.procs then skips it (css_task_iter_advance()).
  • The pids controller uncharges a task in pids_release(), its .release callback, which cgroup_release() calls from release_task(), once the zombie's parent collects it.

Validation

On a Linux x86-64 host, in nvx-rs, with NVX_BUNDLE=skip and Rust 1.95.0 unless noted:

  • cargo fmt --all --check
  • cargo clippy --locked --all-targets --all-features -- -D warnings
  • cargo clippy --locked --lib -- -D warnings
  • cargo clippy --locked --lib --no-default-features -- -D warnings
  • cargo test --locked --all-features --lib -- guest::spawn guest::cgroup guest::exec, then cargo test --locked --all-features
  • the new test 50 times in a row, and the guest::spawn tests 5 times
  • RUSTDOCFLAGS='-D warnings' cargo doc --locked --no-deps --all-features
  • cargo +1.89 check --locked --all-targets --all-features
  • python3 scripts/nvx.py check-guest-binaries, from the repository's root: the musl Clippy and tests of the guest library
  • the new test with the child's cgroup stage moved before its barrier stage: it fails, as it should

On Windows: python scripts\nvx.py verify.

No VM backend ran: the change is documentation and one unit test. CI runs the rest of check-nvx-rs, check-aci-edge-sandboxes, and the KVM, MSHV and WHP jobs.

…unts

A child that Command::cgroup places joins its cgroup in one of its own
launch stages, which spawn does not wait for, and which follows the
release of its barrier, if it waits at one. Until its launch report
arrives, the cgroup may not hold the child, so an occupancy check, a
kill, a removal or a settlement in that window can miss it. Say so on
Command::cgroup, with the two ways to have the child in the cgroup
first, and point to it from ExecutionCgroup::placement, is_occupied and
kill, as remove already does.

The execution binding settles on ExecutionCgroup::is_occupied, which
reads pids.current. pids.current counts a zombie until its parent
collects it, while cgroup.procs and the populated entry of cgroup.events
drop a process as soon as it exits, so a caller that watches those files
can see the cgroup empty before the execution settles. Say so in the
binding's module documentation.

Pin the order that the documentation now states with a test: a child
spawned with a barrier and a cgroup has not joined the cgroup while it
waits at its barrier, and has joined once its launch report arrives.

Nothing else changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e861ffac-4aa0-4c85-9638-6aa98a9c48a3
Copilot AI balanced review requested due to automatic review settings October 27, 2026 03:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The documentation matches the implementation, the regression test checks the stated ordering, and no blocking issues remain.

No open issues 🎉

What changed in this PR

This PR clarifies the guest library’s cgroup placement and settlement guarantees without changing runtime behavior, supporting the broader cgroup work in #912.

Changes:

  • Documents that child-side cgroup placement follows barrier release and is not awaited by spawn.
  • Explains that settlement uses pids.current, which counts zombies until reaped.
  • Adds a regression test verifying cgroup placement occurs after barrier release and before the launch report arrives.
File Description
nvx-rs/​src/​guest/​spawn.rs Documents placement timing and adds a barrier-ordering regression test.
nvx-rs/​src/​guest/​exec/​binding.rs Clarifies settlement accounting and how it differs from live-process indicators.
nvx-rs/​src/​guest/​cgroup.rs Documents placement timing and its effect on occupancy checks and kills.

Review effort: Balanced

@esaurez
Enrique Saurez (esaurez) marked this pull request as ready for review October 27, 2026 03:07
@esaurez
Enrique Saurez (esaurez) deleted the esaurez/exec-cgroup-docs-20261026 branch November 3, 2026 22:06
@esaurez
Enrique Saurez (esaurez) restored the esaurez/exec-cgroup-docs-20261026 branch November 3, 2026 22:21
@esaurez
Enrique Saurez (esaurez) changed the base branch from dev to feature/nvx-refactor November 6, 2026 00:36
@esaurez
Enrique Saurez (esaurez) changed the base branch from feature/nvx-refactor to dev November 6, 2026 02:21
@ppenna Pedro Henrique Penna (ppenna) added the wontfix This will not be worked on label Nov 6, 2026
@ppenna
Pedro Henrique Penna (ppenna) deleted the esaurez/exec-cgroup-docs-20261026 branch November 6, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

wontfix This will not be worked on

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants