In Keras, how do I change a lambda layer after the model is compiled?
More specifically, let's say I want a lambda layer that computes y=a*x+b with a and b being changed every epoch.
import keras
from keras.layers import Input, Lambda, Dense
import numpy as np
np.random.seed(seed=42)
a = 1
b = 2
def f(x, a, b):
return a * x + b
inputs = keras.layers.Input(shape=(3,))
lam = Lambda(f, arguments={"a": a, "b": b})(inputs)
out = keras.layers.Dense(5)(lam)
model = keras.models.Model(inputs, out)
model.trainable = False
model.compile(optimizer='rmsprop', loss='mse')
x1 = np.random.random((10, 3))
x2 = np.random.random((10, 5))
model.fit(x1, x2, epochs=1)
print("Updating. But that won't work")
a = 10
b = 20
model.fit(x1, x2, epochs=1)
This returns twice loss: 5.2914 where it should return once loss: 5.2914 and then loss: 562.0562.
As far as I can tell, this seems to be an open issue, which could be remedied by writing a custom layer but I haven't been to make it work.
Any guidance is welcomed.
If you work with a and b being tensors, you can change their values even after compilation.
There are two approaches. In one, you treat a and b as global vars and take them from outside the function:
import keras.backend as K
a = K.variable([1])
b = K.variable([2])
def f(x):
return a*x + b #see the vars coming from outside here
#....
lam = Lambda(f)(inputs)
At any moment, you can manually call K.set_value(a,[newNumber]).
K.set_value(a,[10])
K.set_value(b,[20])
model.fit(x1,x2,epochs=1)
In another approach (I don't know if there are advantages, but...it sounds at least better organized) you can make a and b be inputs to the model:
a = K.variable([1])
b = K.variable([2])
aInput = Input(tensor=a)
bInput = Input(tensor=b)
def f(x):
return x[0]*x[1] + x[2] #here, we input all tensors in the function
#.....
lam = Lambda(f)([inputs,aInput,bInput])
You set the values of a and b the same way as the other approach.