Do we specifically need cuDNN v5.1 (as suggested) for TensorFlow, or would the latest version (v6.0) work as well? Is there backward compatibility in cuDNN versions?
No, cuDNN 6.0 is not supported in the latest 1.2 version. But there is a hope: the official release notes tell the following:
TensorFlow 1.2 may be the last time we build with cuDNN 5.1. Starting with TensorFlow 1.3, we will try to build all our prebuilt binaries with cuDNN 6.0. While we will try to keep our source code compatible with cuDNN 5.1, it will be best effort.
So hopefully the next 1.3 version will use cuDNN 6.0. Especially now, when 7.0 is right around the corner.