some values of reconstructed images turn in to zero

Question

I'm new in NN field and trying to use autoencoder for image compression using keras library. most of the time I get some zero values in reconstructed vector that cause some black dot in reconstructed images, the zero values increase by decreasing the bottleneck layer neuron numbers. Is this normal ? here is my code :

x = np.load('GrayRef.npy')
x = np.float32(x)

x_train, x_test = train_test_split(x , random_state=104, test_size=0.25, shuffle=True)
x_train = x_train/255 
x_test = x_test/255

EncoderInput = layers.Input(shape=(16, 16))

Flatting = layers.Flatten()(EncoderInput)
Encoder_dense_layer1 = layers.Dense(128, 'sigmoid')(Flatting)
Encoder_dense_layer2 = layers.Dense(64, 'relu')(Encoder_dense_layer1)
Encoder_dense_layer3 = layers.Dense(32, 'relu')(Encoder_dense_layer2)
Encoder = Model(EncoderInput, Encoder_dense_layer3)

Decoder_Input = layers.Input(shape=(32))
Decoder_dense_layer1 = layers.Dense(64, 'relu')(Decoder_Input)
Decoder_dense_layer2 = layers.Dense(128, 'relu')(Decoder_dense_layer1)
Decoder_dense_layer3 = layers.Dense(256, 'relu')(Decoder_dense_layer2)
Shaping = layers.Reshape((16, 16))(Decoder_dense_layer3)
Decoder = Model(Decoder_Input ,Shaping)

AE_Input = layers.Input(shape=(16, 16))
AE_encoder_output = Encoder(AE_Input)
AE_decoder_output = Decoder(AE_encoder_output)

AE = Model(AE_Input, AE_decoder_output)

AE.compile(loss = 'mse', optimizer = optimizers.Adam())

AE.fit(x_train, x_train, epochs=160, batch_size=60, shuffle=True, validation_data=(x_test, x_test))

jj = AE.predict(test)

jj = jj*255

np.save('Test',jj)

the other problem I have is I get different MSE values different time I train the model with same data. for example one time I get 0.0019 and another time I GET 0.04, Is this only because random initial values or there is something I'm missing.

Answer 1

There are some things that come to my mind:

1. Why do you use sigmoid function for encoding? Your Input data is scaled to be between 0 and 1 and I am not sure why it is useful to force the output of the first layer to be between 0 and 1 (which is the purpose of sigmoid activation functions) -> try reLU
2. I can image that you have trouble with your data type (the output is float and you usually want images as np.uint8). This values should be between 0 and 255. HERE it makes sense to use a simgoid function (last layer!) and now, multiplying with 255 (as you do) will lead to floats in the desired range. NOW you can safely cast to integers without getting under or overflows (which might create your "holes")
3. The randomness can be avoided by setting the random seeds for numpy (includes keras) and tensorflow as such (depending on your version, untested)

# first lines in your code!!!
from numpy.random import seed
seed(41)

from tensorflow import set_random_seed
set_random_seed(42)