Search code examples
tensorflowneural-networksom

Are there any difference initializing Self Organizing Map with random distribution and initialize using the first input to the network?


When initialize SOM with random distribution the network may converge correctly but not, when initialize with the input. Why ?


Solution

  • Poor weight initialization can lead to entanglement in the neurons, and becomes a problem when it comes to topological mapping. The problem is when two neurons that should be far apart have ended up representing the same cluster of input data.

    Using your first input as the initialization could be leading to this problem. However, a relatively easy check is to use Sammon Mapping to reduce the dimensions of the clustering nodes into a 2 dimensional representation of the distance between each other. This can be visualized as the nodes with lines connecting adjacent pairs. An unstable learning process can then be discerned from a Sammon map that folds over on itself.

    Sammon Map

    Sammon Map with folds

    This doesn't mean initializing the weights with input data is a bad idea, however I would recommend using random input data as the initialization with the use of something like numpy.random.seed(), as using input data could speed up the learning process.