I have a docker container that acts as a single member of a replica set. I'm able to successfully start three of these up and run some shell scripts inside of them in order to connect them together as a replica set. I've found that I can connect to each of the members individually from my host machine, however when I try to connect to the replica set (in my case, via spring-data-mongodb) by listing each member at the boot2docker IP and respective port, I find that somewhere the application exchanges the boot2docker IP for each containers IP and tries to connect directly at that IP.
I'm assuming this happens because the initial connection string is just a seed list and then the actual configuration is provided by communication with the members themselves. This, of course, is a problem because those IPs do not exist on the host machine (where the app is running), only inside the boot2docker VM. How can I make the members report the boot2docker IP instead of their individual unreachable IPs?
I've come across something similar with elasticsearch and was able to fix it by setting the network.publish_host variable. Is there a similar setting for mongodb? I thought perhaps bind_ip was the correct thing but have been unable to make that work, am I using that incorrectly? I had been setting it to the boot2docker IP and was receiving a bind error on startup.
Also, I can confirm this is only an issue with boot2docker thrown into the mix as my friends running ubuntu and native docker are able to connect just fine.
Here's the error I'm seeing when trying to set my bind_ip to the boot2docker vm:
Starting mongod process using command: /usr/bin/mongod --storageEngine wiredTiger --auth --dbpath /data/db/ --replSet rs0 --bind_ip 127.0.0.1,192.168.59.103 --keyFile /mongodb-keyfile
2015-04-19T16:34:41.123+0000 E NETWORK [initandlisten] listen(): bind() failed errno:99 Cannot assign requested address for socket: 192.168.59.103:27017
Here's my connection string in my spring xml:
<mongo:mongo id="mongo" replica-set="192.168.59.103:27017,192.168.59.103:27018,192.168.59.103:27019"/>
Here's the error my app sees (if I just started things without trying to change the bind_ip, etc)
Client view of cluster state is {type=ReplicaSet, servers=[{address=172.17.0.34:27017, type=Unknown, state=Connecting}, {address=172.17.0.35:27017, type=Unknown, state=Connecting}, {address=172.17.0.36:27017, type=Unknown, state=Connecting}]
at com.mongodb.BaseCluster.getServer(BaseCluster.java:82) ~[mongo-java-driver-2.13.0.jar:na]
Those IPs are the IPs of each container, and each container does indeed start up on port 27017, but then I publish their ports on the boot2docker VM as 27017, 27018, and 27019.
I found a solution to this. Essentially, what matters to mongo clients (and in my case, spring data) are the addresses reported by the replica set members themselves. The initial connection string is only a seed list which is traversed until one of the specified members responds to the client with the full details of the replica set configuration.
That means that the actual IPs that are recorded for retry attempts and really all future interaction with the replica set is determined by what the replica set reports back to the client when the client requests its configuration which is determined by the contents of the replica set config document.
Since the replica set was initiated using the docker container IPs, those were the IPs that were recorded in the replica set config document and then reported to the app on startup and so once the client switched to use those IPs it could no longer connect to the containers because the docker IPs aren't exposed outside of the boot2docker VM.
The solution (in my case) was to change the replica set initial configuration to instead provide the member IPs as though it was being initiated from the host machine (and so really they're just different ports exposed on the boot2docker machine). This meant that the addresses in the config document were reachable by any clients on my host machine and not only those inside boot2docker.