Search code examples
cudaprofilingnvprof

nvprof option for bandwidth

What is the correct option for measuring bandwidth using nvprof --metrics from the command line? I am using flop_dp_efficiency to get the percentage of peak FLOPS, but there seems to be many options for bandwidth measurement in the manual that I don't really understand what I am measuring. e.g. dram_read, dram_write, gld_read, gld_write all look the same to me. Also, should I report bandwdith as a sum of read+write throughput by assuming both happen simultaneously ?

Edit:

Based on the excellent answer with the diagram, what would be the bandwidth going from the device memory to the kernel ? I am thinking to take the minimum of the bandwidth (read+write) on the path from the kernel to the device memory, which is probably dram to L2 cache.

I am trying to determine if a kernel is compute- or memory- bound by measuring FLOPS and bandwidth.

Solution

  • In order to understand the profiler metrics in this area, it's necessary to have an understanding of the memory model in a GPU. I find this diagram (which used to be published in the Nsight Visual Studio edition documentation) to be useful (a similar chart is now part of the nsight compute documentation). I have marked up the diagram with numbered arrows which refer to the numbered metrics (and direction of transfer) I have listed below:

    enter image description here

    Please refer to the CUDA profiler metrics reference for a description of each metric:

    1. dram_read_throughput, dram_read_transactions
    2. dram_write_throughput, dram_write_transactions
    3. sysmem_read_throughput, sysmem_read_transactions
    4. sysmem_write_throughput, sysmem_write_transactions
    5. l2_l1_read_transactions, l2_l1_read_throughput
    6. l2_l1_write_transactions, l2_l1_write_throughput
    7. l2_tex_read_transactions, l2_texture_read_throughput
    8. texture is read-only, there are no transactions possible on this path
    9. shared_load_throughput, shared_load_transactions
    10. shared_store_throughput, shared_store_transactions
    11. l1_cache_local_hit_rate
    12. l1 is write-through cache, so there are no (independent) metrics for this path - refer to other local metrics
    13. l1_cache_global_hit_rate
    14. see note on 12
    15. gld_efficiency, gld_throughput, gld_transactions
    16. gst_efficiency, gst_throughput, gst_transactions

    Notes:

    1. An arrow from right to left indicates read activity. An arrow from left to right indicates write activity.
    2. "global" is a logical space. It refers to a logical address space from the programmers point of view. Transactions directed to the "global" space could end up in one of the caches, in sysmem, or in device memory (dram). "dram", on the other hand, is a physical entity (as is the L1 and L2 caches, for example). The "logical spaces" are all depicted in the first column of the diagram immediately to the right of the "kernel" column. The remaining columns to the right are physical entities or resources.
    3. I have not tried to mark every possible memory metric with a location on the chart. Hopefully this chart will be instructive if you need to figure out the others.

    With the above description, it's possible your question still may not be answered. It would then be necessary for you to clarify your request -- "what do you want to measure exactly?" However based on your question as written, you probably want to look at the dram_xxx metrics, if what you care about is actual consumed memory bandwidth.

    Also, if you are simply trying to get an estimate of the maximum available memory bandwidth, using the CUDA sample code bandwidthTest is probably the easiest way to get a proxy measurement for that. Just use the reported device to device bandwidth number, as an estimate of the maximum memory bandwidth available to your code.

    Combining the above ideas, the dram_utilization metric gives a scaled result that represents the portion (from 0 to 10) of the total available memory bandwidth that was actually used.