This question is about the blktrace tool. On several Ubuntu 3.16.0 machines in our lab I need to trace the software vs device block IO performance. We sometimes use our custom nvme driver and sometimes the standard one. Here is a excerpt of the blkparse output (with the standard nvme driver):
259,0 2 189505 9.997188463 8160 Q R 126875648 + 248 [fio]
259,0 2 189506 9.997191290 8160 Q R 126875896 + 8 [fio]
259,0 2 189507 9.997215574 8160 Q R 363057152 + 248 [fio]
259,0 2 189508 9.997218444 8160 Q R 363057400 + 8 [fio]
259,0 2 189509 9.997219210 8160 C R 216536568 + 8 [0]
259,0 2 189510 9.997220497 8160 C R 126875896 + 8 [0]
259,0 2 189511 9.997230160 8160 C R 363057400 + 8 [0]
259,0 2 189512 9.997248050 8160 Q R 147316736 + 248 [fio]
259,0 2 189513 9.997250930 8160 Q R 147316984 + 8 [fio]
259,0 2 189514 9.997277161 0 C R 147316984 + 8 [0]
This shows the Queued and Complete actions but not the D - issued actions that I am interested in. That is the problem. I need more actions (events) shown. This is from
blktrace /dev/nvme0n1
meanwhile, on other Linux machines it works, or even on the same machine if I trace a different device like
blktrace /dev/sda
That works as shown in this excerpt:
8,0 18 69 17.778827207 8538 Q RA 306186592 + 8 [ls]
8,0 18 70 17.778827767 8538 G RA 306186592 + 8 [ls]
8,0 18 71 17.778828037 8538 I R 306186592 + 8 [ls]
8,0 18 72 17.778828284 8538 D R 306186592 + 8 [ls]
8,0 18 73 17.778832181 8538 A RA 306186600 + 8 <- (8,1) 306184552
8,0 18 74 17.778832397 8538 Q RA 306186600 + 8 [ls]
8,0 18 75 17.778832951 8538 G RA 306186600 + 8 [ls]
8,0 18 76 17.778833221 8538 I R 306186600 + 8 [ls]
8,0 18 77 17.778833441 8538 D R 306186600 + 8 [ls]
8,0 18 78 17.778837161 8538 A RA 306186608 + 8 <- (8,1) 306184560
This last one (with /dev/sda) shows all the different actions, which is great. So how do I get the detailed blktrace for the nvme0n1 device? And why does it not automatically show the other actions (besides Q and C)?
You should be able to see D as long as your nvme device is has a block device interface. Try libaio engine in fio and run a random write. Make sure to set NOOP IO scheduler in /sys/block/nvme../scheduler