Search code examples
linuxembeddedmmappcisysfs

Mapping CompactPCI device through sysfs-pci driver

So, the problem can be described as follows:

  1. We got 11 completely equal PCI devices, connected through two CompactPCI buses, 6 on one, and 5 on the other.
  2. We are trying to access the resources of the devices through the sysfs filesystem, example: /sys/class/pci_bus/0000:04/device/0000:04:0d.0/resource1. First 4 devices allow read/write access to their resources without problems, but:
  3. The 5th and 6th devices of both buses don't work: all files exist, but all read operations return a bunch of FFs, regardless of the written values, so I can't really say if the write was successful or not. When one of the first 4 is physically removed, 5th device starts working as usual, same goes for 6 on the bus with 6 devices. It looks like it can only work with 4 devices per bus, not more. It should be noted that CompactPCI allows using 7 PCI devices on the bus at once, according to the specification.
  4. It can't really be a hardware problem, because Windows driver(developed long ago by someone we don't have access to) does it just fine.

lspci:

03:0b.0 Multimedia controller: Device 6472:8001 (rev 01)
03:0c.0 Multimedia controller: Device 6472:8001 (rev 01)
03:0d.0 Multimedia controller: Device 6472:8001 (rev 01)
03:0e.0 Multimedia controller: Device 6472:8001 (rev 01)
03:0f.0 Multimedia controller: Device 6472:8001 (rev 01)
04:09.0 Multimedia controller: Device 6472:8001 (rev 01)
04:0a.0 Multimedia controller: Device 6472:8001 (rev 01)
04:0b.0 Multimedia controller: Device 6472:8001 (rev 01)
04:0c.0 Multimedia controller: Device 6472:8001 (rev 01)
04:0d.0 Multimedia controller: Device 6472:8001 (rev 01)
04:0f.0 Multimedia controller: Device 6472:8001 (rev 01)

lspci -vv(equal aside from bus numbers for all 11 devices):

04:0f.0 Multimedia controller: Device 6472:8001 (rev 01)
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx+
Interrupt: pin A routed to IRQ 11
Region 0: I/O ports at d800 [size=128]
Region 1: Memory at febfe800 (32-bit, non-prefetchable) [size=128]

Don't know if I really need show you the code, because it is as simple as it is possible - file is opened, then mmaped, then the resulting pointer is used to write and read into that file.

fd = open ( (device_ + "resource" + std::to_string (i)).c_str(), O_RDWR);
ptr = (u_int32_t*) mmap (NULL, 0x7f, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

All paths are recovered right, that's what I've checked first.

dmesg has no errors regarding PCI.

Solution

  • After quite a long time, I've decided to answer this question. I didn't solve the problem by myself, and wrote an email to maintainer of PCI-related code in linux kernel. After tens of attempts to find out what went wrong, we just stopped - I had to switch to another project, spare time was over. The only thing that has been is discovered is that in such a configuration you CANNOT use mmap(and this is the primary way of accessing BARs through sysfs filesystem). So, instead, I have developed a simple PCI driver, which does exactly the same thing, but using read/write operations, and it worked.

    Basically, 

    kernel -> userspace - result

ioremap -> read/write - works
ioremap -> mmap - doesn't work
sysfs -> mmap - doesn't work