Search code examples
pythonpandastensorflowmaskingzero-padding

Tensor-flow how to use padding and masking layer in case of MLPs?

I want to use MLPs to solve regression problem.

I have inputs with variable length to fix this I want to use Zero-padding with masking layer.

I read the inputs from csv file using pandas library. Here is how my data look like.

image1: how train data look like on my csv

I know only how to fill NaN values with 0 using this command x_train.fillna(0.0).values

Like the first row :

[4, 0, 0, 512, 1.0, 0.0, 1.0, 0.0, 128.0 , NaN]

After padding :

[4, 0, 0, 512, 1.0, 0.0, 1.0, 0.0, 128.0 , 0.0]

The mask should be like this :

[1, 1, 1, 1, 1, 1, 1, 1, 1, 0]

But I do not know how to add the mask layer and feed them into my MLPs.

If i have fixed input length. My program will look like this

...
n_input = 10 #number og inputs

train_X = pd.read_csv('x_train.csv')
train_Y = pd.read_csv('y_train.csv')


X = tf.placeholder("float", [None, n_input])
Y = tf.placeholder("float", [None, n_output])

...
y_pred = multilayer_perceptron(X)
...

with tf.Session() as sess:
    sess.run(init)

            _, c = sess.run([train, loss], feed_dict={X: train_X,
                                                      Y: train_Y})
          ...

I do not know how to combine between Zero padding and masking layer?

Solution

  • You cannot ignore a single Features in an MLP. Mathematically we are talking about a matrix multiplication. The only dimensions you can "ignore" are time dimensions in recurrent layers since the number of weights does not scale with the dimension of time and so a single layer can take different sizes in the time dimension.

    If you are only using Dense layers you cannot skip anything because your only dimension (besides the batch dimensions) scales directly with the number of weights.