feat: implement virtio based host-guest communication - #1717
Conversation
Add distinct VecBytes and ByteChunks function value types while preserving embedded stack encoding. Introduce the external-byte marker for future virtq codecs and simplify the virtq message header. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Add external-aware function call and result encoding and decoding. This changes are transitional only and next commits will unify embedded and external (de)coding paths. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Read received payloads directly into caller owned storage and require paired completion of readable/writable chains. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Make pool restoration transactional. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Split and rename the pool implementations, add explicit lower/upper SlotPool regions. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Define deterministic producer and consumer reset behavior. Validate canonical events, descriptor chains, IDs, buffer policy, and unused descriptors. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Represent scratch bookkeeping with one repr(C) layout. Derive offsets and assert the host/guest ABI at compile time. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Use the first GPA of the reserved pages as an exclusive limit. Accept allocations ending at the limit and reject address overflow. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Define directional queue depths, buffer sizes, and pool page counts. Account for guest allocated rings and pools in minimum scratch calculations. Publish the transport contract through scratch-top metadata. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
This patch adds guest owned G2H and H2G rings and pools during guest initialization, and prefill H2G receive capacity. The guest then publishes their gpas through scratch metadata. The patch is also validates allocation order, scratch ownership, and canonical ring images before installing either host consumer. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Persist validated G2H and H2G ring images in running snapshots. Restore fixed transport allocations and install fresh host consumers before sandbox execution. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Initialize queues before first guest entry. Carry host calls, logs, external values, and bounded responses over G2H chains. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Route guest calls, host calls, results, and logs through bidirectional packed virtqueues. Stream dense messages across exact chain batches and reserve H2G capacity for retained external values. Keep external byte returns typed through guest dispatch and support owner-backed byte chunks in Rust and C guests. Store canonical rings in a versioned OCI transport layer and validate them during snapshot load and restore. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Enter the guest once before capture to canonicalize both queues. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Report retained pool slots through the checkpoint mailbox. Keep rejected sandboxes usable for release and retry. Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Use queue_size instead of queue_depth Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
andreiltd
added
the
kind/enhancement
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
andreiltd
added
regen-goldens
What are the implications? Is there any behavior changes for existing hyperlight consumers? |
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
There is no behavior changes for existing consumers, there is a new parameter type that is zero-copy: Adding snapshot support for retained The only real change for existing customers is minimum scratch and guest heap size increase as they need to account for virtual queues bookkeeping. |
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
Allocate logical regions atomically Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
This still will cause OOM as the test requires but will increase initial heap size so that virtqueus finish initialization.
Signed-off-by: Tomasz Andrzejak <andreiltd@gmail.com>
ludfjig
left a comment
ludfjig
left a comment
There was a problem hiding this comment.
I haven't looked very detailed through the code yet:
You mentioned offline that separating ByteChunks into a separate PR without the virtio stuff is hard because ByteChunks dependo on virtio. Would it be possible instead to have the first pr be virtio stuff, and second be the bytechunks (just to make it easier to review)?
It's also my understanding that the new parameter type vec<Bytes> (rather than just Bytes without vec) is necessary for the guest to receive data zerocopy as it's not contiguous memory, but this type is also mirrored in the host api. I guess my question is do we expect it to be a useful for a host to provide multiple Bytes rather than just 1 contiguous one?
It looks like hlbytechunks and hlsizeprefixedbytechunks are not used anymore after recent commits, should they be removed?
| pub(crate) fn is_compatible_with(&self, other: &Self) -> bool { | ||
| // Exhaustive destructure so adding a field to | ||
| // `SandboxMemoryLayout` fails to compile here, forcing the | ||
| // author to decide whether it participates in compatibility. | ||
| let Self { | ||
| input_data_size, | ||
| output_data_size, | ||
| heap_size, | ||
| code_size, | ||
| init_data_size, | ||
| init_data_permissions, | ||
| scratch_size, | ||
| g2h_queue_size, | ||
| h2g_queue_size, | ||
| g2h_buffer_size, | ||
| h2g_buffer_size, | ||
| g2h_pool_pages, | ||
| h2g_pool_pages, | ||
| snapshot_size: _, | ||
| pt_size: _, | ||
| } = self; | ||
| *input_data_size == other.input_data_size | ||
| && *output_data_size == other.output_data_size | ||
| && *heap_size == other.heap_size | ||
| *heap_size == other.heap_size | ||
| && *code_size == other.code_size | ||
| && *init_data_size == other.init_data_size | ||
| && *init_data_permissions == other.init_data_permissions | ||
| && *scratch_size == other.scratch_size | ||
| && *g2h_queue_size == other.g2h_queue_size | ||
| && *h2g_queue_size == other.h2g_queue_size | ||
| && *g2h_buffer_size == other.g2h_buffer_size | ||
| && *h2g_buffer_size == other.h2g_buffer_size | ||
| && *g2h_pool_pages == other.g2h_pool_pages | ||
| && *h2g_pool_pages == other.h2g_pool_pages | ||
| } |
There was a problem hiding this comment.
just fyi I have a draft PR #1728 that will relax this requiement, and use the target snapshot as authority, requiring nothing from the sandbox's layout
See rendered design doc. For broader context see HIP PR: #1112
This patch replaces stack based communication with function calls transported over virtual queues. The documentation covers two current limitations: snapshots do not support retained buffers, and the host owns the transport arena. Addressing both is deferred to future work.