Search code examples
jenkinshigh-availabilitynfs

Jenkins: 2 master nodes using NFS


I´m thinking about the following high availability solution for my enviroment:

  • Datacenter with one powered on Jenkins master node.

  • Datacenter for desasters with one off Jenkins master node.

Datacenter one is always powered on, the second is only for disasters. My idea is install the two jenkins using the same ip but with a shared NFS. If the first has fallen, the second starts with the same ip and I still having my service successfully

My question is, can this solution work?.

Thanks all by the hekp ;)


Solution

  • I don't see any challenges as such why it should not work. But you still got to monitor in case of switch-over because I have faced the situation where jobs that were running when jenkins abruptly shuts down were still in the queue when service was recovered but they never completed afterwards, I had to manually delete the build using script console.

    Over the jenkins forum a lot of people have reported such bugs, most of them seems to have fixed, but still there are cases where this might happen, and it is because every time jenkins is restarted/started the configuration is reloaded from the disk. So there is inconsistency at times because of in memory config that were there earlier and reloaded config.

    So in your case, it might happen that your executor thread would still be blocked when service is recovered. Thus you got to make sure that everything is running fine after recovery.