We have a 10 thousand row table that has just 2 columns, a primary key and a second column that keep state. The problem is that we need this state to be replicated across 3 physical locations in the US(about 2000 miles apart), in near real time or as fast as practically possible over a network. Any of the 3 locations can update the state for a given row in this table which should be replicated in near real time to the other 2 locations.
Are there any open source or commercial light weight in-memory database that could help us achieve what we are trying to do. Disk persistence is not that important here.
Check out Redis. Here's the Replication Howto.
Also, if you decide that the DB doesn't absolutely need to be in-memory, it just needs to be fast, you might want to consider CouchDB. It can do continuous replication, which is essentially instant, and all nodes are masters. It has a well-thought-out conflict detection and resolution mechanism. This blog post is a great introduction to the latest and greatest CouchDB replication capabilities.