VFIO PCIe BAR write won't work. Register value falls back when program execution finished

Question

I am using the following code to test a PCIe BAR register write through VFIO_PCI APIs.

`   struct vfio_group_status group_status = { .argsz = sizeof(group_status) };
    struct vfio_device_info device_info = { .argsz = sizeof(device_info) };
    struct vfio_region_info reg = { .argsz = sizeof(reg) };
    int container = open("/dev/vfio/vfio", O_RDWR);
    int group = open("/dev/vfio/17", O_RDWR);

    /* Add the group to the container */
    ioctl(group, VFIO_GROUP_SET_CONTAINER, &container);
    /* Enable the IOMMU model we want */
    ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_IOMMU);

    int dev = ioctl(group, VFIO_GROUP_GET_DEVICE_FD, "0000:02:00.0");

    void* ptr;
    ptr = mmap(0, 4096, PROT_READ | PROT_WRITE, MAP_SHARED, dev, ( 0UL << 40));
    cout << *((unsigned int *)ptr + 0x202) << endl; //this always print out 0
    *((unsigned int *)ptr + 0x202) = 4; //write 4
    cout << *((unsigned int *)ptr + 0x202) << endl;` //this always print out 4

However, the write will take effect only if the program is still running. After the program execution finished, the value always falls back.

I have verified this is not because of cache, because I can see the register value change in the FPGA while the program is still running.

My question is: what can I do to make my write persistent?

Answer 1

For any one who might encouter with this in the future.

I found this related post. Apparently, this is an expected behavior: if VFIO-PCI is used to control a PCIe device. "vfio_pci_try_bus_reset" will be called upon a process finished. Causing the device to be reset. As mentioned in the post, it would be nice if a module parameter can be added to by pass the reset. But so far it seems that the parameter is not there yet.