We have 2 web servers and 1 app server in our QA environment. Each of them hosts a Membase service. For simplicity let's call the web servers: web01 and web02. Both running IIS.
To access Membase we use the .Net client: http://memcached.enyim.com/
The problem we're encountering here is that web01 can access Membase, set and get values without any issues, any and every time. However web02 can't seem to access Membase from our web application.
We wrote a test console app and ran it from the web02 server and it worked just fine.
We've even tried to Telnet into the url and port we have in the web02 config and that worked.
The Membase configuration values are definied in the machine.config file. We compared our machine.config files between web01 and web02 w/ WinMerge and they're identical.
Then I created a dump of our MembaseClient object using reflection and here is what we get on Web02:
==========================================
Setting Value: 1337047585
On the Key: testPageKey_287945569
"Setting NOT successful!!"
*Membase Client dump:*
bucketName: OurBucketName
bucketPassword: ThisIsAPassword
poolUrls count: 3
Url_0: http://web01:8091/pools/OurBucketName
Url_1: http://web02:8091/pools/OurBucketName
Url_2: http://app01:8091/pools/OurBucketName
Node: 69.147.125.66:11210 is alive: False
Node: 69.147.125.67:11210 is alive: False
Node: 69.147.125.68:11210 is alive: False
==========================================
Please note how the last 3 lines mention Node IsAlive = False... This is very puzzling to me because web01 says all the nodes are alive and so does the Membase Admin Console. And again from our test app, running from anywhere, including web02, we can set and get values successfully in that QA Membase cluster.
Anyone has ever encountered this issue? Or has any idea about where we should be looking to solve it?
Thanks.
Found the issue. It was related to the .Net install on the servers. Not all of them had the required .Net 3.5 SP1 install. You need SP1 for the Enyim library to work properly.
See: http://www.couchbase.org/forums/thread/machine-cant-connect-cluster