Skip to content

Merge dev into master: VirtualBox driver - #365

Open
Wenzel wants to merge 114 commits into
masterfrom
dev
Open

Wenzel wants to merge 114 commits into
masterfrom
dev

Conversation

@Wenzel

@Wenzel Wenzel commented Oct 31, 2026 •

Copy link
Copy Markdown
Owner

Merges the VirtualBox driver work from dev into master: 107 commits, 74 files, +9320/−450. dev is rebased onto master (merge commits kept), so it now includes the pyo3 0.29 and setuptools 83 bumps (#267, #268).

What's included

PRs merged into dev: #293, #296–#311, #323. The commits since #323 went to dev directly (events, memory views and the test/doc series up to cb859f0).

Before merging

  • CI must pass on this PR.
  • The VirtualBox live tests need a VBoxVMI build of Wenzel/virtualbox vmi/vboxvmi and don't run in CI.

🤖 Generated with Claude Code

Wenzel and others added 30 commits October 11, 2026 20:09
VBoxInitParams gains MachineDebugger { vm_name }, behind the new
virtualbox-api feature (vboxapi crate, by path). It reads guest physical
memory through IMachineDebugger::readPhysicalMemory, and reports the end
of the highest guest RAM range as max physical address, so RAM above
4 GiB is covered. Without VBoxInitParams the FDP access method is used as
before (virtualbox feature, unchanged).

The examples take --vbox_machinedebugger <vm_name>. Driver docs describe
both access methods. Also fixes the DriverInitParams doctest (stray ///).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
- vboxapi is now a git dependency on Wenzel/rs-vboxapi (pinned rev),
  so the build no longer needs a ../vboxapi checkout.
- The VM is named by common.vm_name for every access method (KVM
  convention); VBoxInitParams::MachineDebugger carries no name, and the
  examples take --vm_name <name> --vbox_machinedebugger.
- Selecting an access method that isn't compiled returns the new
  MicrovmiError::AccessMethodNotCompiled naming the feature to enable,
  instead of DriverNotCompiled.
- read_physical reports partial progress like the Xen driver: on
  failure, bytes_read is the readable prefix up to the first bad page.
- CI builds and lints virtualbox-api in the Linux VirtualBox job.
- Docs: pause/resume aren't implemented yet (#277), how to build
  against a local vboxapi checkout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
virtualbox: MachineDebugger access method, physical read
Python: VBoxInitParamsPy enum (MachineDebugger()) set on
DriverInitParamsPy.virtualbox; pymicrovmi gains the virtualbox-api
feature passthrough. mem-dump.py gains --vbox-machinedebugger and
--no-pause.

C: DriverInitParamsFFI.virtualbox (VBoxInitParamsFFI) with an
access_method tag; zero-initialized selects the default (FDP).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
- C: VBoxInitParamsFFI.access_method is a plain u32 with
  VBOX_ACCESS_METHOD_NONE / VBOX_ACCESS_METHOD_MACHINE_DEBUGGER
  constants (a C-filled Rust enum is UB for out-of-range values).
  Unknown values, like invalid strings, are now reported through
  microvmi_init's init_error instead of panicking.
- build.rs: rerun cbindgen when anything under src/ changes.
- C example vbox-read-page (cmake and Makefile).
- Volatility handler: vbox_access_method=machinedebugger query param.
- mem-dump.py: pass DriverType.VirtualBox with --vbox-machinedebugger.
- CI: smoke-import VBoxInitParamsPy, build wheels with virtualbox-api.
- Docs: Python API, Volatility parameters, docstrings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
bindings: VirtualBox access method selection from Python and C
The VirtualBox context restores a snapshot, starts the VM headless and
creates the driver for one access method (MachineDebugger for now; the
VBoxVMI method will reuse it). Configured from the environment
(VBOX_APP_HOME, VBOX_TEST_VM, VBOX_TEST_SNAPSHOT, VBOX_TEST_MEMORY_MB to
cold boot with more RAM, e.g. above 4 GiB).

First VirtualBox test: with the VM paused, every guest page of a
dumpvmcore core file below the max physical address reads the same
through the driver.

The harness now takes a test-name filter, a MICROVMI_TEST_TIMEOUT
override and exits non-zero when a test fails.

Closes #276

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
- read test: compare exactly the guest's RAM ranges (debugvm info phys
  verbose), fail on any unreadable RAM page or one missing from the
  core, check the RAM above 4 GiB; use the driver's bytes_read
- harness: MICROVMI_TEST_FILTER instead of a positional argument, fail
  when it matches nothing; teardown also runs when setup panics; abort
  the run on timeout, since the test thread can't be stopped
- VirtualBox context: timeout scaled with guest RAM, vCPU count from
  showvminfo for get_vcpu_count, core path from VBOX_TEST_CORE_DIR and
  deleted at teardown, restore waits for transient VM states and reports
  VBoxManage's last error
- gate the VirtualBox tests like init_context does

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
tests: VirtualBox context in the integration harness
VBoxInitParams::VBoxVMI { socket_path } selects it, behind the new
virtualbox-vmi feature (Linux only). Init connects to the VM's VBoxVMI
socket with the vboxvmi crate and completes the handshake; the other
Introspectable methods come with #282-#285. common.vm_name is ignored by
this method: the socket belongs to one VM.

The examples take --vbox_vmi <socket>; CI builds and lints the feature
in the Linux VirtualBox job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
…ad of panicking

Phase-1 VM state access methods of the VBoxVMI driver return
NotImplemented until #282-#285 add the commands. Also bump vboxvmi to
a9562e9 (reused receive buffer) and document the socket directory
requirements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
virtualbox: VBoxVMI access method (connect and handshake)
Pause and resume go through IConsole (VirtualBox's own pause, visible
to the GUI and VBoxManage), synchronous and idempotent. The vCPU count
is the VM's IMachine::CPUCount instead of unimplemented.

Tests: VBOX_TEST_CPUS cold boots the guest with that many vCPUs, and
virtualbox_pause_idempotent_and_visible checks the pause semantics
against showvminfo. The --no-pause notes in the driver docs are gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
- vboxapi 473ab32: pause/resume succeed only in the target state (VmBusy
  during a live snapshot or teleport), retry races with other clients.
- tests: discard the saved state once when both VBOX_TEST_MEMORY_MB and
  VBOX_TEST_CPUS are set.
- tests: virtualbox_vcpu_count_matches_vmm checks the count against the
  VMM's per-vCPU statistics; virtualbox_pause_stops_vcpus checks every
  vCPU's executing time is frozen while paused and grows after resume.
- docs: resume undoes pauses by other clients, busy states, CPU hot-plug
  limitation, VBOX_TEST_CPUS=2 for the multi-vCPU checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
virtualbox: pause/resume and vCPU count for the MachineDebugger method
MachineDebugger: rs-vboxapi e8b3d20 probes memory access in Vm::attach,
so a VirtualBox without the IMachineDebugger memory access patch fails
init with an error naming the patch, not the first read.

init(None, ..): when init_params.virtualbox is set, the caller targets
VirtualBox, so its driver error is returned instead of NoDriverAvailable.

tests/virtualbox_errors.rs covers VBoxVMI socket missing (explicit and
autodetect), protocol version mismatch and bad magic with a fake server,
and the unpatched VirtualBox case (#[ignore], run with VBOX_APP_HOME set
to a stock build). CI runs the VBoxVMI ones and lints the tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
virtualbox: clear errors for misconfigured VirtualBox setups
VBoxApi::write_physical calls vboxapi's Vm::write_physical
(IMachineDebugger::writePhysicalMemory), pinned to rs-vboxapi a8b54f9.
Not atomic: on error a prefix may be written, never a non-RAM page.

New test virtualbox_write_physical_matches_core, method-agnostic: with
the VM paused, it writes a seeded random pattern to zero pages (Linux's
reserved low 64 KiB, middle of RAM, top below 4 GiB, top of RAM), checks
it reads back, is in a fresh core where the rest of RAM is unchanged
(so the rejected MMIO and above-RAM writes wrote nothing), survives a
resume/re-pause cycle, and is zeroed back (checked in a last core).

Cores are now unlinked once open, so two can coexist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BuBbUKAMBjAEvmVqbjRkmd
CI: sync Rust 1.99 fixes from master into dev
…address from RAM ranges

Physical reads go through batched PHYS_READ requests (several ranges per
request, a status per page); unreadable pages are reported through
bytes_read without failing the call, and the readable prefix survives a
later transport or protocol error. The max physical address comes from
RAM_RANGES.

Python and C bindings can select the VBoxVMI access method. mem-dump
derives the name from the socket and requires --no-pause for VBoxVMI;
Volatility rejects vbox_socket_path unless method=vboxvmi.

Test harness: the default method follows the compiled features, setup
records and teardown restores VBoxInternal/VMI/Enabled, and the socket
path is resolved like VirtualBox does (VBOX_USER_HOME, legacy
~/.VirtualBox, XDG_CONFIG_HOME rooted at $HOME when relative,
~/.config). The context is built with VirtualBox::new.
virtualbox: write physical memory through the MachineDebugger method
pause and resume go through the VBoxVMI PAUSE and RESUME commands:
VirtualBox's standard pause, so the VM shows as Paused in VBoxManage and
the GUI, synchronous (every vCPU stopped when pause returns) and
idempotent (one resume undoes any number of pauses, by anyone). The vCPU
count comes from VCPU_COUNT at connect, like the max physical address.
rs-vboxvmi is bumped to 622c077 for the new commands.

mem-dump.py no longer requires --no-pause for VBoxVMI. The driver and
test docs list pause/resume and vCPU count as supported, and the VBoxVMI
test filters cover the pause, resume and vCPU tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Wenzel added 22 commits October 31, 2026 08:32
…-rate page access

The VBoxVMI tests assumed one vCPU: an int3 written at the idle loop runs on
every vCPU, and the processes started for CR3 writes run wherever the guest
schedules them. The breakpoint, emulated-int3, emulate-insn, fast-single-step,
memory-view and CR3 tests now set their Intercept on every vCPU, take the first
vCPU that stops and release the others. An idle vCPU can halt for over a
second, so a vCPU counts as running once it executes within 5 s.

New tests cover the memory view lifecycle and 15-view limit, several views,
restoring a Remap, Remaps on SMP, exhausting the Hidden pool and 3 s of
high-rate page access changes (which found a VirtualBox guru meditation, fixed
in 2cd860d69eb). The reset and refused-save tests now start from a session that
holds a view, a Remap, the Hidden pool, a restriction and a parked vCPU.

Harness: setup sets VBoxInternal/VMI/HiddenPages for every method, and unsets
VMI/Enabled for the non-VBoxVMI selections so the default method really falls
back. The RAM-range parsing leaves out the Hidden pool. VBoxManage is retried
while VBoxSVC is shutting down, the time limit has 40 s more for VBoxManage
calls, and teardown powers off a VM stuck in a guru meditation. The driver
reference gets a test-suite section pointing at tests.md.

Refs Wenzel/drakvuf#19
The MachineDebugger method counted the VBoxVMI Hidden page pool as guest
RAM (max physical address 0x20010000 on a 512 MiB guest with 16 Hidden
pages), so virtualbox_read_physical_matches_core failed. The fix is in
rs-vboxapi 1f5b3de; patch the git dependency to the local clone until it
is pushed, like the kvmi patch.

Refs Wenzel/drakvuf#32
…tream

Both reproduce on an unpatched build of the VBoxVMI fork point
(32f5f1de3fe) at the same rates as on the patched build (BIOS hang
7/30 vs 6/30 guest reboots, clocksource warning 2/36 vs 1/40 boots),
so users hitting them need the workaround, not a VBoxVMI fix.

Refs Wenzel/drakvuf#31
Refs Wenzel/drakvuf#28
The VirtualBox, rs-vboxvmi and libmicrovmi histories were rewritten into
atomic commits, so the build and measurement references in the driver
reference and ADR 0008 named commits that no longer exist on any branch.
Each now names the rewritten commit that contains it; for the commits
that absorbed later fix-ups, that is the nearest equivalent rather than
an identical tree (mapping on Wenzel/drakvuf#25).

Refs Wenzel/drakvuf#25
…w and unallocated targets

Fails on VirtualBox 2cd860d69eb (an unallocated Hidden page was accepted
as a Remap target), passes on fea349d1d62 on 2 vCPUs.

Refs #358
…X_TEST_IEM

vboxvmi_memory_view_restricted_stack restricts vCPU 0's idle stack page in
a view and expects the interrupt frame push to be reported in that view.
It failed on VT-x and with VBOX_TEST_IEM (all guest code in VirtualBox's
IEM, VBoxInternal/EM/IemExecutesAll) before VirtualBox e52e897b6c0, and passes
in both after it.

Refs #359
Protocol version 12, the CPUID and #DB intercepts, the CSTAR, SFMASK and
KERNEL_GS_BASE registers, and DRAKVUF's -F, which works on VirtualBox;
only -p, ipt, hidsim and dirwatch are Xen-only.

Refs Wenzel/drakvuf#25
…isconnect

vboxvmi_memory_view_restrictions_per_view: one page restricted
differently in two views keeps both, a change leaves the other view, the
refusals of the Default view apply, and only the view that denies the
access reports it (with event.view). vboxvmi_memory_view_disconnect_
clears_restrictions: a normal close leaves no restriction in the next
client's view. Both pass on base and smp2.

Refs #355
…t is killed

vboxvmi_hidden_pool_disconnect allocates the whole pool, closes, and the
next client allocates it all again; then a child process takes the pool
and is killed, and the pool is whole once more.

Refs #357
vboxvmi_memory_view_reply_switch: vCPU 0's single-step reply moves it
into a view whose idle-loop copy has an int3; vCPU 0 alone stops there
(in that view), the other vCPUs idle on in Default. A reply into a view
that doesn't exist is refused and the vCPU stays parked for a valid
reply. Passes on base and smp2.

Refs #356
…efused

The page's real bytes are read back after the POP got the supplied data,
and a 257-byte EmulateReadData is refused with the vCPU still parked for
the reply that follows.

Refs #348
vboxvmi_events_singlestep_breakpoint_recoil repeats DRAKVUF's recoil five
times on one int3 (restore, step on, reply; step event; re-arm, step off)
and checks that stepping onto an int3 still in place reports the
breakpoint, not a step. Passes on base and smp2. ADR 0011 records that a
stepping vCPU takes no external interrupt (the reversed criterion of
#329), and the driver page says single step needs VT-x with MTF.

Refs #329
vboxvmi_memory_view_emulate_through_remap gives only the Remap target an
int3 at the faulting instruction; Emulate must run it (breakpoint event at
the source gpa, in that view). Failed with VERR_EM_INTERPRETER before
VirtualBox 0529e7053cd, passes on base and smp2 after it.

Refs #359
vboxvmi_memory_view_emulate_write_through_remap remaps vCPU 0's idle stack
page in a view to a copy and write-restricts it there; an emulated stack
write changes the copy and leaves the source page's bytes alone. Passes on
base (3 runs) and smp2.

Refs #359
vboxvmi_memory_view_emulate_read_through_remap: at a POP from vCPU 0's
idle stack, remapped in a view to a copy and denied read and write there,
only the copy holds a marker at the read address; Emulate loads the
marker. Passes on base (2 runs) and smp2.

Refs #359
Add one symbol-free example targeting MSR_LSTAR or an explicit GPA. The page-access method emulates reads, writes and the replaced instruction; the memory-view method executes a patched Hidden-page copy and steps accesses through the Default view. Both report per-vCPU hits and restore all state on count or Ctrl-C stop.\n\nLive tests make the guest CPU read and overwrite an int3-patched byte and verify the original read value, logical write and retained int3 for both methods on one and two vCPUs.\n\nRefs #352\nRefs #360
Restore the exact pre-POP register state after an emulated read through a Remap, execute the same POP natively through EPT, and require the same value, RIP and RSP.\n\nAlso require refused Remaps to preserve their source and target pages, and refused emulation to preserve both pages and the parked vCPU. The comparison and refused-emulation checks pass on base and smp2; Remap refusals pass on base.\n\nRefs #359
Kill a client after it leaves an int3-backed Remap, then prove a fresh session reuses a clean view and all vCPUs resume.

Refs #358
Wenzel and others added 7 commits October 31, 2026 08:34
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nb2AnDh3QKD5xTvTRaXoK2
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Nb2AnDh3QKD5xTvTRaXoK2

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant