VMware DirectPath I/O - adding passthrough PCI devices to VM

VMware DirectPath I/O allows guest OS to directly access physical PCI and PCIe devices connected to host. There several things to check before proceeding to configuration:

• the maximum of PCI devices that can be attached to a VM is 6
• the VM hardware must be 7 or later
• Intel VT-d or AMD IOMMU must be enabled
• PCI devices are connected and marked as available 

In vSphere Client  go to Configuration - Advanced Settings (Hardware) and see if the the device is Active (a green icon). If there is no device displayed, go to Edit and select your device. In some cases the host will need reboot - device will have an orange icon.

In /etc/vmware/esx.conf the modification is recorded as:
/device/000:000:27.0/owner = "passthru"

Go to VM - Edit Settings and add the device

A reservation equal to the memory of the VM will be automatically created. However, the reservation is not removed when the device is removed from the VM. So, be sure to clean up. If there is no memory reservation then powering on the VM will end in the following error:

Another way to do add/remove passthrough devices is using PowerCLI. The  add-passthroughdevice cmdlet does not create the memory reservation and it has to be done in a second step:

get-vmhost HostName | get-passthroughdevice | Where {$_.State -eq "Active"} | add-PassThroughDevice -VM VMName

foreach ($vm in (get-vm VMName)) {get-vmresourceconfiguration $vm | set-vmresourceconfiguration -MemReservationMB $vm.MemoryMB}

Removal of PCI device and memory reservation cleanup:

get-vm VMName | get-passthroughdevice | remove-passthroughdevice -Confirm:$false

get-vm VMName |get-vmresourceconfiguration | set-vmresourceconfiguration -MemReservationMB 0

A VM using DirectPath I/O does not support the following features:

• snapshots
• suspend and resume
• HA
• FT
• DRS (the VM can be part of DRS cluster, but it cannot be migrated across hosts)
• hot adding and removal of devices