Search code examples
tensorflowkerasdeep-learningautoencoder

Tensorflow variational autoencoder, how is decoder connected?


In the keras example at https://blog.keras.io/building-autoencoders-in-keras.html, the section "Variational autoencoder (VAE)" writes

z = layers.Lambda(sampling)([z_mean, z_log_sigma])
# Create encoder
encoder = keras.Model(inputs, [z_mean, z_log_sigma, z], name='encoder')

# Create decoder
latent_inputs = keras.Input(shape=(latent_dim,), name='z_sampling')
x = layers.Dense(intermediate_dim, activation='relu')(latent_inputs)
outputs = layers.Dense(original_dim, activation='sigmoid')(x)
decoder = keras.Model(latent_inputs, outputs, name='decoder')

I am confused which layer is connected to latent_inputs and where does it define the connection? For example, in vae.fit(), x_train is passed as VAE input and goes to the layer inputs, which is defined as inputs = keras.Input(shape=(original_dim,)) and input of the VAE model is defined as vae = keras.Model(inputs, outputs, name='vae_mlp').


Solution

  • You're missing one important part of the tutorial:

    # instantiate VAE model
    outputs = decoder(encoder(inputs)[2])
    vae = keras.Model(inputs, outputs, name='vae_mlp')
    

    In Keras, a Model can be used as a Layer in an other Model. (Source)

    So the output z of the encoder is fed as the input of the decoder to form the VAE model.