usbvfiod

usbvfiod is a Rust-based tool designed to enable USB device passthrough to Cloud Hypervisor virtual machines using the vfio-user protocol. Other VMMs might also work, but but are currently not the main target.

This project is still under active development and not usable yet. We are planning to work on this project in the following order:

1. Validating our Assumptions (Done)
   • We are looking for suitable libraries to use and finalize our design.
2. Towards USB Storage Passthrough (🚧 Ongoing 🚧)
   • We build up a virtual XHCI controller and the necessary plumbing to pass-through USB devices from the host.
   • Our initial test target will be USB storage devices.
3. Broaden Device Support
   • We broaden the set of USB devices we support and actively test.

If you want to use this code, please check back later or get in touch, if you need professional support.

Documentation

Find the overview of documentation here.

Development

The following section is meant for developers.

Testing with Cloud Hypervisor

An easy way to get a testing setup is to connect usbvfiod with Cloud Hypervisor. For this, start usbvfiod in one terminal:

$ cargo run -- --socket-path /tmp/usbvfiod.sock -vv
2025-04-25T09:41:40.891734Z  INFO usbvfiod: We're up!

In another terminal, start Cloud Hypervisor. Any recent version will do:

$ cloud-hypervisor \
   --memory size=4G,shared=on \
   --serial tty \
   --user-device socket=/tmp/usbvfiod.sock \
   --console off \
   --kernel KERNEL \
   --initramfs INITRD \
   --cmdline KERNEL_CMDLINE

KERNEL, INITRD, and KERNEL_CMDLINE are placeholders for a Linux kernel image (bzImage), a initrd or initramfs and the corresponding command line.

Tip

To get a kernel and initramfs to play with, you can use the NixOS netboot binaries.

You will find a kernel (bzImage) and initrd. The required command line for booting is in result/netboot.ipxe. You want to add console=ttyS0 to get console output.

$ nix-build -A netboot.x86_64-linux '<nixpkgs/nixos/release.nix>'
$ ls result/
bzImage initrd netboot.ipxe
...
$ grep -o "init=[^$]*" result/netboot.ipxe
init=/nix/store/.../init initrd=initrd nohibernate loglevel=4

Attaching USB Devices

For the time being, USB devices can only be attached when usbvfiod is started. usbvfiod takes the path to the USB device node. These paths are of the form /dev/bus/usb/$BUS/$DEVICE.

To figure out the bus and device numbers of a specific USB device, use the lsusb utility (typically installed via the usbutils package):

$ lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:0033 Intel Corp. AX211 Bluetooth
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 003: ID 18a5:0243 Verbatim, Ltd Flash Drive (Store'n'Go)

So for attaching the flash drive you would add --device /dev/bus/usb/002/003 as a parameter to usbvfiod. usbvfiod must have permission to read and write the device node.

Note

Attached USB devices do not yet appear in the guest. The relevant plumbing is not implemented yet.

Format Checks

.toml files in the repository are formatted using taplo. To re-format .toml files, you can use:

$ taplo format file.toml

Temporarily Ignoring Pre-Commit Checks

When committing incomplete or work-in-progress changes, the pre-commit checks can become annoying. In this case, use:

$ git commit --no-verify

Kernel Debug Messages for XHCI

The XHCI driver in the Linux kernel prints helpful messages with xhci_dbg. If your kernel supports CONFIG_DYNAMIC_DEBUG (the NixOS netboot kernel does), you can enable the messages for XHCI with:

$ echo "file drivers/usb/host/xhci* +p" | sudo tee /sys/kernel/debug/dynamic_debug/control

Then, you can filter the kernel log for the relevant messages:

$ sudo dmesg | grep -E xhci\|usb

Tip

You can also enable the dynamic debug messages on boot by adding xhci_hcd.dyndbg==pmfl xhci_pci.dyndbg==pmfl to the command line.

Alternatively, provide dyndbg==pfml as option to modprobe on invocation or through a modprobe config.