Multithreaded backpropagation

Question

I have written a back propagation class in VB.NET -it works well- and I'm using it in a C# artificial intelligence project.

But I have a AMD Phenom X3 at home and a Intel i5 at school. and my neural network is not multi-threaded.

How to convert that back propagation class to a multithreaded algorithm? or how to use GPGPU programming in it? or should I use any third party libraries that have a multithreaded back propagation neural network?

Answer 1

JeffHeaton has recommend that you use resilient propagation (RPROP) instead of backpropagation. There are examples on how to do multithreaded RPROP (MPROP):

• Article on C# multithreaded backpropagation (from Jeff heaton)
• Chapter 7.2.1- "Propagation and Multithreading" (p.94 of Introduction to Encog 2.5 for C#)

It's a difficult to discuss all of the details here, so I would recommend that you either read that article and take a look at the relevant chapters of the book I referenced. This, of course, is assuming you're familiar with concurrent programming.

Update:

Resilient propagation will typically outperform backpropagation by a considerable factor. Additionally, RPROP has no parameters that must be set. Backpropagation requires that a learning rate and momentum value be specified. Finding an optimal learning rate and momentum value for backpropagation can be difficult. This is not necessary with resilient propagation. (source: Encog Machine Learning)