For example can I have Atlas, OpenBlas, MKL installed on my Ubuntu 14.04 at the same time and switch between them without recompiling Caffe?
Sure, you have to have them installed, and on Ubuntu/Debian issue command
update-alternatives --config libblas.so.3
You'll get numbered list of alternatives and could switch between them easily
Link: https://wiki.debian.org/DebianScience/LinearAlgebraLibraries