Search code examples
deep-learningnoiseautoencoder

Where to add noise in the Stacked Denoising Autoencoder?

For a Stacked Denoising Autoencoder as following Figure original figure are from link

I was wondering where to add noise? For a single layer denoising autoencoder, we only add noise to the input.

For multi-layer denoising autoencoder, do we need to add noise at the position 1,2,3,4 in the figure, or we only need to add noise in the position 1?

Thanks

Solution

  • The figure is not showing a stacked autoencoder -- it is rather a "deeper" autoencoder with a single encoder (consisting of an intermediate layer), a single code/latent vector/hidden layer and a single decoder(also with an intermediate layer).

    A stacked denoising autoencoder is just the same as a stacked autoencoder but you replace each layer’s autoencoder with a denoising autoencoder while you keep the rest of the architecture the same.

    It is important to mention that in each layer you are trying to reconstruct the autoencoder's previous input - added with some noise which you can think of simply as corrupting the input.

    For all subsequent layers, you are basically reconstructing the output (i.e. hidden layer) of the autoencoder in the previous layer.

    Also, to answer your question - you will need to add noise in each and every layer.

    More information can be found here: http://psyyz10.github.io/2015/11/SDA/