Search code examples
c++cntk

disabling GPU makes my CNTK program work. Why?


I have a C++ program that uses CNTK and works fine on dozens of PCs. But on this one particular PC, it crashes when loading the neural networks from disk -- at program startup. Finally, I disable the GPU (from windows, not from any CNTK code) and then it loads, runs, and works fine. So, why would a GPU being present cause this issue -- as far as I know, I'm not using GPU in my CNTK code. Got an idea? Thanks!


Solution

  • Answering my own question: I checked my code and then noticed/recalled that the LoadModel call takes a DeviceDescriptor as an argument. For some of my networks I had that set to DefaultDevice, which was GPU. I changed those to CPUDevice instead -- and the problem went away. So, it must be that something is 'wrong' with this particular GPU -- since the older version of my code has been fine on all kinds of machines. In my case, I don't need GPU at eval time, so it seems it will be safer to make sure I just specify CPU in my run-time code, just in case other systems have a 'GPU problem' like this PC has (still don't understand the issue, really)