I would like to modify a MacOS X USB XHCI Controller driver. According to Apple, they strongly recommend using a two machine setup for debugging and testing. Unfortunately, my development machine has the target hardware and I don't have access to another suitable machine.
After giving it some thought, would it be possible to create and run a virtual machine for this purpose? Specifically, can I have an OS X guest with direct IO access to my development host machine?
There are really a few questions here, so let's unpack the various parts:
1. macOS kext development/testing/debugging in a Virtual Machine
This generally works pretty well. You can get kprintf logging output if you attach a virtual serial port to the VM, and depending on the emulated ethernet device, you can attach an lldb kernel debugger session. I know this works with VMWare Fusion and Parallels Desktop. I haven't used VirtualBox for a number of years, but last I tried you had to use the virtio network device (and driver) as the way it emulated the default intel ethernet adapter seemed to be incompatible with the OSX's driver's kernel debugging support. They may have fixed this.
If you boot the host machine into Linux, you can also run macOS inside a Qemu/KVM virtual machine; kernel debugging likewise works there. (You can also run it in Qemu on macOS, but that only supports emulation mode which is painfully slow compared to hardware-assisted virtualisation.)
2. Device driver development/testing/debugging in a virtual machine
Either you're writing a driver for a device the VM environment simulates - this case is fairly obvious, of course you can do that. Many VM environments can simulate XHCI host controllers.
You're asking about access to physical host hardware, though, which is typically referred to as pass-through. Whether or not this is possible depends heavily on the virtualisation environment and the type of device:
- USB device pass-through is typically well-supported. All of the virtualisation environments I've already mentioned support this in some form or another. It works better for some types of USB device than others, and a device that doesn't work well via one VM system might work better via another. This type of passthrough happens at the USB bus level however - the host controller sits one level deeper than this though.
- PCI(e) device pass-through is also possible in principle on modern hardware, if a number of components come together. You need:
- A host system which has an active hardware IOMMU. (I/O memory-management unit) Intel's implementation of this is called "VT-d". Macs typically have this from Intel's "Ivy Bridge" Core series CPU onwards. (Core i?-3??? and newer) I believe some older Mac Pros (Xeon) also have this feature. Unlike most PCs, it is enabled by default in firmware on Macs. This is needed so that the guest-physical memory addresses can be transparently passed to the device for DMA purposes - these addresses won't match the actual host-physical memory page addresses, and the IOMMU can translate between the two. Without this, the guest VM could issue DMA read/writes to host memory it doesn't own!
- Host operating system support for the IOMMU. macOS uses it by default if it is available, primarily to defend against malicious Thunderbolt devices. It typically needs to be activated via the kernel command line on Linux; I don't know what the situation is on Windows.
- VM Software that can make use of the IOMMU for PCI device passthrough to guests.
- Qemu/KVM can do this on Linux hosts via the vfio (host) kernel module. I have some personal experience with this, including macOS guests; it works surprisingly well in many cases, even with some GPUs.
- VMWare ESXi apparently also supports PCI passthrough. (supposedly if ESXi is running on Mac hardware, this includes support for macOS guests) I have no personal experience with this setup.
- Common desktop virtualisation environments on macOS hosts (VMWare Fusion, Parallels Desktop, VirtualBox) do not to my knowledge support PCI pass-through.
- I've done some research into the feasibility of supporting PCI passthrough from macOS hosts to xhyve/hyperkit VM guests. I think I know how to do it, but I haven't had the time try implementing it. (And nobody has paid me to try yet either 😉) Even with that, two more hurdles there for your use case would be lack of simulated framebuffer device and lack of macOS guest support in xhyve/hyperkit. Both of those are comparatively straightforward to solve given enough developer resources, however.
In summary:
If you want to do this right now, check your Mac has an IOMMU, and if it does, it would appear you have two options: boot Linux on it and use Qemu/KVM, or use VMWare ESXi; in both cases, create some macOS virtual machines (probably one for testing/debugging, the other for developing/building your kexts) and use vfio to pass the device you want to develop a driver for to the testing VM.
Whether or not this is more convenient than using two Macs is debatable!