RNN with Embedding

Question

Based on this methodology, I was trying to build RNN model with categorical and continuous variable.

The continous placeholder is in this form:

x = tf.placeholder(tf.float32, [None,num_steps, input_size], name="input_x")`

And the categorical data placeholder is in this form:

store, v_store = len(np.unique(data_df.Store.values)), 50

z_store = tf.placeholder(tf.int32, [None, num_steps], name='Store')

emb_store = tf.Variable(
    tf.random_uniform((store, v_store), -r_range, r_range),
    name="store"
    )

embed_store = tf.nn.embedding_lookup(emb_store, z_store)

Finally, I'm concatenating categorical and continuous placeholder together.

inputs_with_embed = tf.concat([x, embed_store], axis=2, name="inputs_with_embed")

This is where I'm multiplying the tensor vector with last layer.

val = tf.transpose(val, [1, 0, 2])
last = tf.gather(val, int(val.get_shape()[0]) - 1, name="lstm_state")
ws = tf.Variable(tf.truncated_normal([lstm_size, input_size]), name="w")
bias = tf.Variable(tf.constant(0.1, shape=[input_size]), name="b")

Edit: All the tensorflow graph code ran fine. But when I was executing the session code, I was getting the following error:

InvalidArgumentError (see above for traceback): Incompatible shapes: [50,4] vs. [50,7,1]
   [[Node: sub = Sub[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](add, _arg_input_y_0_4)]]

And my prediction part.

loss = tf.reduce_mean(tf.square(pred - y), name="loss_mse_train")

Edit end

Can someone please tell me where I'm making the mistake?

Thanks!

Answer 1

As I said, if you want to give each time step a predictive value, you should change ws to [lstm_size, 7] and bias to [7].

ws = tf.Variable(tf.truncated_normal([lstm_size, 7]), name="w")
bias = tf.Variable(tf.constant(0.1, shape=[7]), name="b")

# need to change shape when pred=(?,7) and y=(?,7,1) 
loss = tf.reduce_mean(tf.square(pred - tf.squeeze(y)), name="loss_mse_train")