Skip to content

Commit eda49e1

Browse files
ppennaCopilot
andcommitted
protocol: add NVXC v2's frame codec beside v1
NVXC v1 buffers an execution's input and caps its collected output, so it cannot carry interactive or unbounded exec streams. SL4.4a (#1128) defines version 2's frames and their codec in NVXC's private modules, beside v1, while every peer stays on v1: SL4.4b serves v2 in the guest loop and the fake OpenVMM, and SL4.4c moves the host to it. protocol::app::v2 holds a frame enum with its encoder and decoder. A frame keeps v1's 24-byte header, with version 2, and fits one outer data record. The host sends PING, EXEC, STOP, CANCEL, MAPS, CAPTURE, ACTIVATE, STDIN and STDIN_END, and the guest READY, STDOUT, STDERR, EXIT, STOPPED, STDIN_ACK, BUSY and ERROR. READY carries the protocol version, then, in answer to ACTIVATE, what Ready reports about the activation. EXEC, MAPS, CAPTURE and ACTIVATE carry v1's payloads, which v1's codecs encode and decode, unchanged. STDIN_END ends an execution's input explicitly, and STDIN_ACK returns 1 to 64 KiB of written bytes to its 64 KiB window of unacknowledged input. STDOUT and STDERR have no total cap, so EXIT's outcomes drop v1's output limit. FEATURES has no kind in v2. Both directions refuse a zero request ID, an empty chunk, an acknowledgement outside the window and an empty error category, and the decoder also refuses another magic or version, unknown kinds, a status outside EXIT and ERROR, and a payload that a kind does not allow. Nothing calls the codec yet, so its module is gated by the features of its users, cli, testing, and guest on Linux x86-64, and carries cfg_attr(not(test), expect(dead_code)) naming SL4.4b and SL4.4c, a test hook for which protocol/app.rs takes one seam in the test-layout allowlist. The codec is crate-private, so the public API doesn't change, and BOOT_CONTRACT_VERSION and CONTROL_CONTRACT_REVISION stay as they are. Its tests pin each frame's bytes and round trip, the protocol version that READY carries, the stdin window and the record's bounds, and the refusals, v1's PING vector among them, check that every frame that decodes encodes back to its bytes, and carry v1's EXEC, MAPS, capture and activate payloads across v2 unchanged and in order. Part of #1128 (SL4.4a). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent f03c21c commit eda49e1

5 files changed

Lines changed: 731 additions & 3 deletions

File tree

‎nvx-rs/README.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -168,7 +168,10 @@ the host mapping table, and the `nvx_maps=` and `nvx_workload_account=create`
168168
tokens. NVX's own control protocol, `NVXC`, is private to the crate, gated by
169169
the features of its users, `cli`, `guest` and `testing`: its application
170170
frames, the `EXEC` payload that the host encodes and the guest agent decodes,
171-
and the control features that a guest image advertises.
171+
and the control features that a guest image advertises. Beside version 1's
172+
frames, `protocol::app::v2` holds the codec of version 2's, which carry every
173+
control path, with version 1's payloads, and stream an execution's standard
174+
input, output and error.
172175

173176
The [conformance vectors](tests/vectors/README.md) of version 1,
174177
`tests/vectors/nvxc-v1.txt`, and OpenVMM's vectors and cases of the `NVXS`

‎nvx-rs/src/protocol/app.rs‎

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
//!
44
//! A frame is a 24-byte header (the magic, the version, the frame kind, two zero flags bytes, the
55
//! request ID, a status, and the payload length) followed by the payload. Responses repeat the ID
6-
//! of their request.
6+
//! of their request. This module holds version 1, and `v2` version 2, which carries every control
7+
//! path and streams an execution's standard input, output and error.
78
89
#[cfg(any(
910
feature = "cli",
@@ -46,6 +47,20 @@ pub(crate) mod workload {
4647
}
4748
}
4849

50+
#[cfg(any(
51+
feature = "cli",
52+
feature = "testing",
53+
all(feature = "guest", target_os = "linux", target_arch = "x86_64")
54+
))]
55+
#[cfg_attr(
56+
not(test),
57+
expect(
58+
dead_code,
59+
reason = "SL4.4b's guest loop and fake OpenVMM serve NVXC v2, and SL4.4c's host speaks it"
60+
)
61+
)]
62+
pub(crate) mod v2;
63+
4964
/// Magic of every application frame.
5065
#[cfg(any(
5166
feature = "cli",

‎nvx-rs/src/protocol/app/v2.rs‎

Lines changed: 334 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,334 @@
1+
//! Version 2 of the application frames (magic `NVXC`), which carries every control path and
2+
//! streams an execution's standard input, output and error.
3+
//!
4+
//! A frame has version 1's 24-byte header, with version [`VERSION`]: the magic, the version, the
5+
//! frame kind, two zero flags bytes, the request ID, a status, and the payload length, followed by
6+
//! the payload. Its integers are little-endian. A frame fits one outer data record, so its
7+
//! payload holds at most [`MAX_PAYLOAD`] bytes. Its request ID is never zero: a request takes a
8+
//! new one, its answers repeat it, and so do the frames of the execution that an `EXEC` starts, in
9+
//! both directions. The status is the exit status of an `EXIT` and the error number of an
10+
//! `ERROR`, and zero in every other frame.
11+
//!
12+
//! | Kind | Code | From | Payload |
13+
//! | --- | --- | --- | --- |
14+
//! | `PING` | `0x01` | Host | None |
15+
//! | `EXEC` | `0x02` | Host | An `EXEC` payload |
16+
//! | `STOP` | `0x03` | Host | None |
17+
//! | `CANCEL` | `0x04` | Host | None |
18+
//! | `MAPS` | `0x06` | Host | A `MAPS` payload |
19+
//! | `CAPTURE` | `0x07` | Host | The capture request's flag |
20+
//! | `ACTIVATE` | `0x08` | Host | The activate request's workload identity and activation record |
21+
//! | `STDIN` | `0x09` | Host | 1 or more bytes of an execution's standard input |
22+
//! | `STDIN_END` | `0x0a` | Host | None |
23+
//! | `READY` | `0x81` | Guest | The protocol version, one byte, then, in answer to `ACTIVATE`, what Ready reports about the activation |
24+
//! | `STDOUT` | `0x82` | Guest | 1 or more bytes of an execution's standard output |
25+
//! | `STDERR` | `0x83` | Guest | 1 or more bytes of an execution's standard error |
26+
//! | `EXIT` | `0x84` | Guest | How the execution ended, an [`Outcome`], as text |
27+
//! | `STOPPED` | `0x85` | Guest | None |
28+
//! | `STDIN_ACK` | `0x86` | Guest | How many more bytes of standard input the guest agent wrote, a `u32` from 1 to [`STDIN_WINDOW`] |
29+
//! | `BUSY` | `0x87` | Guest | None |
30+
//! | `ERROR` | `0xff` | Guest | The error category, such as `invalid-request`, as 1 or more bytes of text |
31+
//!
32+
//! The exchanges of version 1 keep their payloads and their order. `PING` asks whether the guest
33+
//! agent is ready, and `READY` answers with the protocol version, which the host checks. `MAPS`,
34+
//! `CAPTURE` and `ACTIVATE` carry the payloads that their codecs encode and decode, as in
35+
//! version 1, and `READY` acknowledges them; in answer to `ACTIVATE`, it also carries what Ready
36+
//! reports about the activation, in the canonical encoding of
37+
//! [`ReadyActivation`](crate::activation::ReadyActivation). `ERROR` refuses a request, and `BUSY`
38+
//! refuses one while an execution runs. `STOP` shuts the VM down, also while an execution runs,
39+
//! and `STOPPED` acknowledges it.
40+
//!
41+
//! An `EXEC` starts an execution, which its `EXEC` payload describes as in version 1, and which
42+
//! its request ID names until the `EXIT` that ends it, or the `ERROR` that refuses it. Its
43+
//! standard output and error come in `STDOUT` and `STDERR` frames, with no total cap. The host
44+
//! sends its standard input in `STDIN` frames, and ends it with `STDIN_END` after the bytes
45+
//! before it. The guest agent acknowledges the bytes of `STDIN` frames with `STDIN_ACK` once it
46+
//! has written them into the execution's standard input, and the host never has more than
47+
//! [`STDIN_WINDOW`] bytes that the guest agent has not acknowledged. `CANCEL` ends the execution,
48+
//! whose `EXIT` reports it. Version 1's `FEATURES` request, `0x05`, has no place in version 2.
49+
50+
use crate::protocol::{OUTER_MAX_PAYLOAD, ProtocolError, array, violation};
51+
52+
/// Version of the frame format and of the protocol, which each header and each `READY` carry.
53+
pub(crate) const VERSION: u8 = 2;
54+
/// Largest payload of a frame, which fits one outer data record with its header.
55+
pub(crate) const MAX_PAYLOAD: usize = OUTER_MAX_PAYLOAD - HEADER_LEN;
56+
/// Most bytes of an execution's standard input that the host may have sent in `STDIN` frames and
57+
/// the guest agent not yet acknowledged with `STDIN_ACK`.
58+
pub(crate) const STDIN_WINDOW: u32 = 64 * 1024;
59+
60+
/// Length of a frame's header.
61+
const HEADER_LEN: usize = 24;
62+
/// Magic of every frame.
63+
const MAGIC: [u8; 4] = *b"NVXC";
64+
/// Kind of [`Frame::Ping`].
65+
const PING: u8 = 0x01;
66+
/// Kind of [`Frame::Exec`].
67+
const EXEC: u8 = 0x02;
68+
/// Kind of [`Frame::Stop`].
69+
const STOP: u8 = 0x03;
70+
/// Kind of [`Frame::Cancel`].
71+
const CANCEL: u8 = 0x04;
72+
/// Kind of [`Frame::Maps`].
73+
const MAPS: u8 = 0x06;
74+
/// Kind of [`Frame::Capture`].
75+
const CAPTURE: u8 = 0x07;
76+
/// Kind of [`Frame::Activate`].
77+
const ACTIVATE: u8 = 0x08;
78+
/// Kind of [`Frame::Stdin`].
79+
const STDIN: u8 = 0x09;
80+
/// Kind of [`Frame::StdinEnd`].
81+
const STDIN_END: u8 = 0x0a;
82+
/// Kind of [`Frame::Ready`].
83+
const READY: u8 = 0x81;
84+
/// Kind of [`Frame::Stdout`].
85+
const STDOUT: u8 = 0x82;
86+
/// Kind of [`Frame::Stderr`].
87+
const STDERR: u8 = 0x83;
88+
/// Kind of [`Frame::Exit`].
89+
const EXIT: u8 = 0x84;
90+
/// Kind of [`Frame::Stopped`].
91+
const STOPPED: u8 = 0x85;
92+
/// Kind of [`Frame::StdinAck`].
93+
const STDIN_ACK: u8 = 0x86;
94+
/// Kind of [`Frame::Busy`].
95+
const BUSY: u8 = 0x87;
96+
/// Kind of [`Frame::Error`].
97+
const ERROR: u8 = 0xff;
98+
99+
/// A frame of version 2, without its request ID, which [`Frame::encode`] takes and
100+
/// [`Frame::decode`] returns.
101+
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
102+
pub(crate) enum Frame<'a> {
103+
/// `PING`: asks whether the guest agent is ready.
104+
Ping,
105+
/// `EXEC`: starts an execution, which the `EXEC` payload describes.
106+
Exec(&'a [u8]),
107+
/// `STOP`: shuts the VM down.
108+
Stop,
109+
/// `CANCEL`: ends the execution that the request ID names.
110+
Cancel,
111+
/// `MAPS`: hands the guest agent entries of the host mapping table, in a `MAPS` payload.
112+
Maps(&'a [u8]),
113+
/// `CAPTURE`: asks nvx-init to capture the VM, with the capture request's payload.
114+
Capture(&'a [u8]),
115+
/// `ACTIVATE`: asks nvx-init to activate a restored VM, with the activate request's payload.
116+
Activate(&'a [u8]),
117+
/// `STDIN`: bytes of the standard input of the execution that the request ID names.
118+
Stdin(&'a [u8]),
119+
/// `STDIN_END`: ends the standard input of the execution that the request ID names.
120+
StdinEnd,
121+
/// `READY`: acknowledges a request.
122+
Ready {
123+
/// The protocol version that the guest agent speaks.
124+
version: u8,
125+
/// What Ready reports about the activation, in answer to `ACTIVATE`; empty otherwise.
126+
activation: &'a [u8],
127+
},
128+
/// `STDOUT`: bytes of the execution's standard output.
129+
Stdout(&'a [u8]),
130+
/// `STDERR`: bytes of the execution's standard error.
131+
Stderr(&'a [u8]),
132+
/// `EXIT`: ends the execution.
133+
Exit {
134+
/// Its exit status, 128 plus the signal number for [`Outcome::Signal`].
135+
status: i32,
136+
/// How it ended.
137+
outcome: Outcome,
138+
},
139+
/// `STOPPED`: acknowledges `STOP`.
140+
Stopped,
141+
/// `STDIN_ACK`: acknowledges this many more bytes of the execution's standard input, which
142+
/// the guest agent wrote into it.
143+
StdinAck(u32),
144+
/// `BUSY`: refuses a request while an execution runs.
145+
Busy,
146+
/// `ERROR`: refuses a request.
147+
Error {
148+
/// The guest's error number.
149+
status: i32,
150+
/// The error category, such as `invalid-request`.
151+
category: &'a [u8],
152+
},
153+
}
154+
155+
/// How an execution ended, which [`Frame::Exit`] reports. Version 2 caps no output, so no
156+
/// execution ends at an output limit.
157+
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
158+
pub(crate) enum Outcome {
159+
/// The workload exited; the status is its exit status.
160+
Exit,
161+
/// A signal ended the workload; the status is 128 plus the signal number.
162+
Signal,
163+
/// The workload's timeout elapsed, and the guest agent killed it.
164+
Timeout,
165+
/// A cancellation ended the workload.
166+
Cancelled,
167+
/// The guest agent could not supervise the workload.
168+
Failed,
169+
}
170+
171+
impl Outcome {
172+
/// Returns the text that names this outcome in an `EXIT` payload.
173+
fn text(self) -> &'static [u8] {
174+
match self {
175+
Self::Exit => b"exit",
176+
Self::Signal => b"signal",
177+
Self::Timeout => b"timeout",
178+
Self::Cancelled => b"cancelled",
179+
Self::Failed => b"failed",
180+
}
181+
}
182+
183+
/// Returns the outcome that `text` names, if any.
184+
fn parse(text: &[u8]) -> Option<Self> {
185+
[
186+
Self::Exit,
187+
Self::Signal,
188+
Self::Timeout,
189+
Self::Cancelled,
190+
Self::Failed,
191+
]
192+
.into_iter()
193+
.find(|outcome| outcome.text() == text)
194+
}
195+
}
196+
197+
impl<'a> Frame<'a> {
198+
/// Encodes this frame with `request_id`.
199+
///
200+
/// It refuses what [`Frame::decode`] refuses: a zero request ID, an empty chunk of standard
201+
/// input, output or error, an acknowledgement of no bytes or of more than [`STDIN_WINDOW`], an
202+
/// empty error category, and a payload longer than [`MAX_PAYLOAD`].
203+
pub(crate) fn encode(&self, request_id: u64) -> Result<Vec<u8>, ProtocolError> {
204+
self.check(request_id)?;
205+
let ready;
206+
let ack;
207+
let (kind, status, head, data): (u8, i32, &[u8], &[u8]) = match *self {
208+
Self::Ping => (PING, 0, &[], &[]),
209+
Self::Exec(data) => (EXEC, 0, &[], data),
210+
Self::Stop => (STOP, 0, &[], &[]),
211+
Self::Cancel => (CANCEL, 0, &[], &[]),
212+
Self::Maps(data) => (MAPS, 0, &[], data),
213+
Self::Capture(data) => (CAPTURE, 0, &[], data),
214+
Self::Activate(data) => (ACTIVATE, 0, &[], data),
215+
Self::Stdin(data) => (STDIN, 0, &[], data),
216+
Self::StdinEnd => (STDIN_END, 0, &[], &[]),
217+
Self::Ready {
218+
version,
219+
activation,
220+
} => {
221+
ready = [version];
222+
(READY, 0, &ready, activation)
223+
}
224+
Self::Stdout(data) => (STDOUT, 0, &[], data),
225+
Self::Stderr(data) => (STDERR, 0, &[], data),
226+
Self::Exit { status, outcome } => (EXIT, status, &[], outcome.text()),
227+
Self::Stopped => (STOPPED, 0, &[], &[]),
228+
Self::StdinAck(count) => {
229+
ack = count.to_le_bytes();
230+
(STDIN_ACK, 0, &ack, &[])
231+
}
232+
Self::Busy => (BUSY, 0, &[], &[]),
233+
Self::Error { status, category } => (ERROR, status, &[], category),
234+
};
235+
let length = head.len() + data.len();
236+
if length > MAX_PAYLOAD {
237+
return Err(violation("an NVXC frame must fit one outer data record"));
238+
}
239+
let mut frame = Vec::with_capacity(HEADER_LEN + length);
240+
frame.extend_from_slice(&MAGIC);
241+
frame.extend_from_slice(&[VERSION, kind, 0, 0]);
242+
frame.extend_from_slice(&request_id.to_le_bytes());
243+
frame.extend_from_slice(&status.to_le_bytes());
244+
frame.extend_from_slice(&(length as u32).to_le_bytes());
245+
frame.extend_from_slice(head);
246+
frame.extend_from_slice(data);
247+
Ok(frame)
248+
}
249+
250+
/// Decodes a frame, and returns its request ID with it.
251+
///
252+
/// It refuses bytes that hold no whole frame, or more than one outer data record holds; a
253+
/// header with another magic or version, nonzero flags, or a payload length other than the
254+
/// payload's; an unknown kind; a nonzero status outside `EXIT` and `ERROR`; a payload in a
255+
/// kind that has none; a `READY` without a version; a `STDIN_ACK` without exactly a `u32`;
256+
/// an `EXIT` whose text names no [`Outcome`]; and what [`Frame::encode`] refuses.
257+
pub(crate) fn decode(frame: &'a [u8]) -> Result<(u64, Self), ProtocolError> {
258+
let (header, payload) = frame
259+
.split_at_checked(HEADER_LEN)
260+
.filter(|_| frame.len() <= OUTER_MAX_PAYLOAD)
261+
.ok_or_else(malformed)?;
262+
if header[..4] != MAGIC || header[4] != VERSION {
263+
return Err(violation(
264+
"the control peer sent a frame of another protocol than NVXC version 2",
265+
));
266+
}
267+
let request_id = u64::from_le_bytes(array(&header[8..16]));
268+
let status = i32::from_le_bytes(array(&header[16..20]));
269+
let length = u32::from_le_bytes(array(&header[20..24]));
270+
if header[6..8] != [0, 0] || length as usize != payload.len() {
271+
return Err(malformed());
272+
}
273+
let empty = |variant| payload.is_empty().then_some(variant);
274+
let decoded = match header[5] {
275+
PING => empty(Self::Ping),
276+
EXEC => Some(Self::Exec(payload)),
277+
STOP => empty(Self::Stop),
278+
CANCEL => empty(Self::Cancel),
279+
MAPS => Some(Self::Maps(payload)),
280+
CAPTURE => Some(Self::Capture(payload)),
281+
ACTIVATE => Some(Self::Activate(payload)),
282+
STDIN => Some(Self::Stdin(payload)),
283+
STDIN_END => empty(Self::StdinEnd),
284+
READY => payload
285+
.split_first()
286+
.map(|(&version, activation)| Self::Ready {
287+
version,
288+
activation,
289+
}),
290+
STDOUT => Some(Self::Stdout(payload)),
291+
STDERR => Some(Self::Stderr(payload)),
292+
EXIT => Outcome::parse(payload).map(|outcome| Self::Exit { status, outcome }),
293+
STOPPED => empty(Self::Stopped),
294+
STDIN_ACK => payload
295+
.try_into()
296+
.ok()
297+
.map(|count| Self::StdinAck(u32::from_le_bytes(count))),
298+
BUSY => empty(Self::Busy),
299+
ERROR => Some(Self::Error {
300+
status,
301+
category: payload,
302+
}),
303+
_ => None,
304+
}
305+
.filter(|decoded| status == 0 || matches!(decoded, Self::Exit { .. } | Self::Error { .. }))
306+
.ok_or_else(malformed)?;
307+
decoded.check(request_id)?;
308+
Ok((request_id, decoded))
309+
}
310+
311+
/// Checks what both directions refuse of a frame with `request_id`, other than its size: a
312+
/// zero request ID, an empty chunk, an acknowledgement of no bytes or of more than
313+
/// [`STDIN_WINDOW`], and an empty error category.
314+
fn check(&self, request_id: u64) -> Result<(), ProtocolError> {
315+
let valid = match *self {
316+
Self::Stdin(data) | Self::Stdout(data) | Self::Stderr(data) => !data.is_empty(),
317+
Self::StdinAck(count) => (1..=STDIN_WINDOW).contains(&count),
318+
Self::Error { category, .. } => !category.is_empty(),
319+
_ => true,
320+
};
321+
if request_id == 0 || !valid {
322+
return Err(malformed());
323+
}
324+
Ok(())
325+
}
326+
}
327+
328+
/// Returns the error of a frame that version 2 does not allow.
329+
fn malformed() -> ProtocolError {
330+
violation("the NVXC frame of version 2 is malformed")
331+
}
332+
333+
#[cfg(test)]
334+
mod tests;

0 commit comments

Comments
 (0)