Search code examples
c++cudansight

NSight profiler crashes an application

I'm trying to profile the following kernel using NSight Visual Studio profiler:

__global__ void cuMultiplyMatricesStandard(float* A, float* B, float* C, int matrixSize)
{
    int gridsPerMatrixX = (matrixSize + (blockDim.x*gridDim.x) - 1)/(blockDim.x*gridDim.x);
    int gridsPerMatrixY = (matrixSize + (blockDim.y*gridDim.y) - 1)/(blockDim.y*gridDim.y);
    for (int i = 0; i < (gridsPerMatrixX * gridsPerMatrixY); i++)
    {
        int row = (blockIdx.y * blockDim.y + threadIdx.y) + (i/gridsPerMatrixX)*gridDim.y*blockDim.y;
        int col = (blockIdx.x * blockDim.x + threadIdx.x) + (i%gridsPerMatrixX)*gridDim.x*blockDim.x;
        if (row >= matrixSize || col >= matrixSize) continue;
        float Clocal = 0;
        for (int k = 0; k < matrixSize; k++)
            Clocal += A[row*matrixSize + k]*B[k*matrixSize + col];
        C[row*matrixSize + col] = Clocal;
    }
}

The calling code is under http://pastebin.com/kB7c7s9W if it matters.

When I run the application under the NSight debugger without profiling, it works fine ("P" stands for "pass", "F" stands for "fail"):

P P P P P P P
P P P P P P P
P P P P P P P
P P P P P P P
P P P P P P P
P P P P P P P
P P P P P P P
Press any key to continue . . .

When I try to Profile CUDA Application from the "NSIGHT -> Start Performance Analysis" option (using any experiment configuration), though, it crashes:

Nsight: Profiling CUDA Kernel cuMultiplyMatricesStandard on device [0] GeForce GTX 760
Nsight:     Saving Pinned Host Memory     0 allocations    0.0 MB
Nsight:     Saving Device Memory          3 allocations    0.0 MB
Nsight:                           Dependency Analysis ( 1/ 9):.
Nsight:                                 Memory Global ( 2/ 9):
Nsight: Experiments complete, total replays needed:  1
CUDA ERROR IN LINE 83 OF FILE C:/Users/Maciej/Documents/Visual Studio 2012/Projects/CUDA-PR/CUDA-PR/Main.cu: unknown error (30)
CUDA ERROR IN LINE 86 OF FILE C:/Users/Maciej/Documents/Visual Studio 2012/Projects/CUDA-PR/CUDA-PR/Main.cu: unknown error (30)
F F F F F F F
F F F F F F F
F F F F F F F
F F F F F F F
F F F F F F F
F F F F F F F
F F F F F F F
Press any key to continue . . .

Line 83 is the kernel call, line 86 is the subsequent cudaMemcpy. The profiler then complains about not registering any kernel launches, and I get no data. What happened?

Solution

  • Nsight 3.1 had a bug in the CUDA profiler dependency analysis experiment to result in some kernels crashing on the next replay. Updating to a more recent version of Nsight fixes the issue.