I'm working on a project right now where I'm trying to implement the NEAT (Neuroevolution of Augmenting Topologies) methodology to a neural network. This has already been done many times over and there have even been updated revisions to the methodology over the past several years in several languages, so I'm not producing anything new at the moment. My little "twist" to the methodology is to attempt a distributed implementation (preferably through MapReduce jobs, although MPI is also a possible choice). I'm wondering if anyone has ever come across a "good" model of a distributed ANN? I've tried a few different google searches, but have come up with nothing of value.
For MPI you could check this paper Parallel implementations of feed-forward neural network using MPI and C# on .NET platform For map-reduce: proceeding