Search code examples
tensorflowtensorboard

Tensorboard - Splitting a fully connected layer into 2 histograms for vizualization

I am aware that I can visualize the weights of the layers in a histogram using tensorboard Understanding TensorBoard (weight) histograms

My question, is it possible to "split" a fully connected layer into two separate histograms ? Because I have input coming from 2 sources that is concatenated before before going through a fully connected layer and I want to see the weight distribution for the 2 sources. Below I have a simple example where a and b are concatenated before being passed through a fully connected layer.

a is of size 1024 and b of size 256. The out layer has 1024 units.

out = tf.matmul(tf.concat(values=(a, b), axis=1), weight) + bias
Solution

  • Assuming your weight to have shape 1280 x 1024, you can first split your weight as

    weight_a = tf.slice(weight, [0, 0], [1024, 1024])
weight_b = tf.slice(weight, [1024, 0], [1280, 1024])

    Now, you can visualize weight_a and weight_b.

    The slicing can be generalized as well but since you explicitly specified the size of each tensor, the above is the quickest method.