Training different outputs at different epochs

Question

Is it possible in Keras that the training of each or some of outputs in multi-output training start at different epochs? For example one of the outputs takes some other outputs as its input. But those outputs at the beginning are quite premature and it brings huge computational burdens to the model. This output that I would like its training to be postponed to some time later is a custom layer that has to apply some image processing operations to its input which is an image generated by another output but at the beginning that the generated image is quite meaningless, I think it's just waste of time for first epochs to apply this custom layer. Is there a way to do that? Like we have weights over each output's loss, do we have different starting point for calculating each output's loss?

Answer 1

1. Build a model that does not contain the later output.
2. Train that model to the degree you want.
3. Build a new model that incorporates the old model into it.
4. Compile the new model with the new loss functions you want.
5. Train that model.

To elaborate on step 3: Keras models can be used like layers in Keras' functional API.

You can build a normal model like so:

input = Input((100,))
x = Dense(50)(input)
x = Dense(1, activation='sigmoid')(x)
model = Model(input, x)

However, if you have another standard Keras model, it can be used just like any other layer. For example, if we have a model (created with Sequential(), Model(), or keras.models.load_model()) called model1, we can put it in like this:

input = Input((100,))
x = model1(input)
x = Dense(1, activation='sigmoid')(x)
model = Model(input, x)

This would be the equivalent of putting in each layer in model1 individually.