I'm trying to determine the number of 'neurons / nodes' within my Keras network not the parameter. I'm using an already implemented variant, so I didn't develop anything by myself.
That I can get an overview of the network and the number of parameters with summary, I know. The problem here is, I don't want to know how many parameters I have but how many 'neurons'. Background is, for an 8 to 8 fully connected layer, i get 64 parameters. But I want to get to the 16. That the whole story with a Conv2D layer is not so easy to make, I also know.
My first approach was to multiply all values of the output_shape variable and add them afterwards. Can I do that or is it wrong?
thats is the list form model summary:
Layer (type) Output Shape
================================================================
input_image (InputLayer) (None, None, None, 1)
zero_padding2d_1 (ZeroPadding2D) (None, None, None, 1)
conv1 (Conv2D) (None, None, None, 64)
bn_conv1 (BatchNorm) (None, None, None, 64)
activation_1 (Activation) (None, None, None, 64)
max_pooling2d_1 (MaxPooling2D) (None, None, None, 64)
res2a_branch2a (Conv2D) (None, None, None, 64)
bn2a_branch2a (BatchNorm) (None, None, None, 64)
activation_2 (Activation) (None, None, None, 64)
res2a_branch2b (Conv2D) (None, None, None, 64)
bn2a_branch2b (BatchNorm) (None, None, None, 64)
activation_3 (Activation) (None, None, None, 64)
res2a_branch2c (Conv2D) (None, None, None, 256)
res2a_branch1 (Conv2D) (None, None, None, 256)
bn2a_branch2c (BatchNorm) (None, None, None, 256)
bn2a_branch1 (BatchNorm) (None, None, None, 256)
add_1 (Add) (None, None, None, 256)
res2a_out (Activation) (None, None, None, 256)
res2b_branch2a (Conv2D) (None, None, None, 64)
bn2b_branch2a (BatchNorm) (None, None, None, 64)
activation_4 (Activation) (None, None, None, 64)
res2b_branch2b (Conv2D) (None, None, None, 64)
bn2b_branch2b (BatchNorm) (None, None, None, 64)
activation_5 (Activation) (None, None, None, 64)
res2b_branch2c (Conv2D) (None, None, None, 256)
bn2b_branch2c (BatchNorm) (None, None, None, 256)
add_2 (Add) (None, None, None, 256)
res2b_out (Activation) (None, None, None, 256)
res2c_branch2a (Conv2D) (None, None, None, 64)
bn2c_branch2a (BatchNorm) (None, None, None, 64)
activation_6 (Activation) (None, None, None, 64)
res2c_branch2b (Conv2D) (None, None, None, 64)
bn2c_branch2b (BatchNorm) (None, None, None, 64)
activation_7 (Activation) (None, None, None, 64)
res2c_branch2c (Conv2D) (None, None, None, 256)
bn2c_branch2c (BatchNorm) (None, None, None, 256)
add_3 (Add) (None, None, None, 256)
res2c_out (Activation) (None, None, None, 256)
res3a_branch2a (Conv2D) (None, None, None, 128)
bn3a_branch2a (BatchNorm) (None, None, None, 128)
activation_8 (Activation) (None, None, None, 128)
res3a_branch2b (Conv2D) (None, None, None, 128)
bn3a_branch2b (BatchNorm) (None, None, None, 128)
activation_9 (Activation) (None, None, None, 128)
res3a_branch2c (Conv2D) (None, None, None, 512)
res3a_branch1 (Conv2D) (None, None, None, 512)
bn3a_branch2c (BatchNorm) (None, None, None, 512)
bn3a_branch1 (BatchNorm) (None, None, None, 512)
add_4 (Add) (None, None, None, 512)
res3a_out (Activation) (None, None, None, 512)
res3b_branch2a (Conv2D) (None, None, None, 128)
bn3b_branch2a (BatchNorm) (None, None, None, 128)
activation_10 (Activation) (None, None, None, 128)
res3b_branch2b (Conv2D) (None, None, None, 128)
bn3b_branch2b (BatchNorm) (None, None, None, 128)
activation_11 (Activation) (None, None, None, 128)
res3b_branch2c (Conv2D) (None, None, None, 512)
bn3b_branch2c (BatchNorm) (None, None, None, 512)
add_5 (Add) (None, None, None, 512)
res3b_out (Activation) (None, None, None, 512)
res3c_branch2a (Conv2D) (None, None, None, 128)
bn3c_branch2a (BatchNorm) (None, None, None, 128)
activation_12 (Activation) (None, None, None, 128)
res3c_branch2b (Conv2D) (None, None, None, 128)
bn3c_branch2b (BatchNorm) (None, None, None, 128)
activation_13 (Activation) (None, None, None, 128)
res3c_branch2c (Conv2D) (None, None, None, 512)
bn3c_branch2c (BatchNorm) (None, None, None, 512)
add_6 (Add) (None, None, None, 512)
res3c_out (Activation) (None, None, None, 512)
res3d_branch2a (Conv2D) (None, None, None, 128)
bn3d_branch2a (BatchNorm) (None, None, None, 128)
activation_14 (Activation) (None, None, None, 128)
res3d_branch2b (Conv2D) (None, None, None, 128)
bn3d_branch2b (BatchNorm) (None, None, None, 128)
activation_15 (Activation) (None, None, None, 128)
res3d_branch2c (Conv2D) (None, None, None, 512)
bn3d_branch2c (BatchNorm) (None, None, None, 512)
add_7 (Add) (None, None, None, 512)
res3d_out (Activation) (None, None, None, 512)
res4a_branch2a (Conv2D) (None, None, None, 256)
bn4a_branch2a (BatchNorm) (None, None, None, 256)
activation_16 (Activation) (None, None, None, 256)
res4a_branch2b (Conv2D) (None, None, None, 256)
bn4a_branch2b (BatchNorm) (None, None, None, 256)
activation_17 (Activation) (None, None, None, 256)
res4a_branch2c (Conv2D) (None, None, None, 1024)
res4a_branch1 (Conv2D) (None, None, None, 1024)
bn4a_branch2c (BatchNorm) (None, None, None, 1024)
bn4a_branch1 (BatchNorm) (None, None, None, 1024)
add_8 (Add) (None, None, None, 1024)
res4a_out (Activation) (None, None, None, 1024)
res4b_branch2a (Conv2D) (None, None, None, 256)
bn4b_branch2a (BatchNorm) (None, None, None, 256)
activation_18 (Activation) (None, None, None, 256)
res4b_branch2b (Conv2D) (None, None, None, 256)
bn4b_branch2b (BatchNorm) (None, None, None, 256)
activation_19 (Activation) (None, None, None, 256)
res4b_branch2c (Conv2D) (None, None, None, 1024)
bn4b_branch2c (BatchNorm) (None, None, None, 1024)
add_9 (Add) (None, None, None, 1024)
res4b_out (Activation) (None, None, None, 1024)
res4c_branch2a (Conv2D) (None, None, None, 256)
bn4c_branch2a (BatchNorm) (None, None, None, 256)
activation_20 (Activation) (None, None, None, 256)
res4c_branch2b (Conv2D) (None, None, None, 256)
bn4c_branch2b (BatchNorm) (None, None, None, 256)
activation_21 (Activation) (None, None, None, 256)
res4c_branch2c (Conv2D) (None, None, None, 1024)
bn4c_branch2c (BatchNorm) (None, None, None, 1024)
add_10 (Add) (None, None, None, 1024)
res4c_out (Activation) (None, None, None, 1024)
res4d_branch2a (Conv2D) (None, None, None, 256)
bn4d_branch2a (BatchNorm) (None, None, None, 256)
activation_22 (Activation) (None, None, None, 256)
res4d_branch2b (Conv2D) (None, None, None, 256)
bn4d_branch2b (BatchNorm) (None, None, None, 256)
activation_23 (Activation) (None, None, None, 256)
res4d_branch2c (Conv2D) (None, None, None, 1024)
bn4d_branch2c (BatchNorm) (None, None, None, 1024)
add_11 (Add) (None, None, None, 1024)
res4d_out (Activation) (None, None, None, 1024)
res4e_branch2a (Conv2D) (None, None, None, 256)
bn4e_branch2a (BatchNorm) (None, None, None, 256)
activation_24 (Activation) (None, None, None, 256)
res4e_branch2b (Conv2D) (None, None, None, 256)
bn4e_branch2b (BatchNorm) (None, None, None, 256)
activation_25 (Activation) (None, None, None, 256)
res4e_branch2c (Conv2D) (None, None, None, 1024)
bn4e_branch2c (BatchNorm) (None, None, None, 1024)
add_12 (Add) (None, None, None, 1024)
res4e_out (Activation) (None, None, None, 1024)
res4f_branch2a (Conv2D) (None, None, None, 256)
bn4f_branch2a (BatchNorm) (None, None, None, 256)
activation_26 (Activation) (None, None, None, 256)
res4f_branch2b (Conv2D) (None, None, None, 256)
bn4f_branch2b (BatchNorm) (None, None, None, 256)
activation_27 (Activation) (None, None, None, 256)
res4f_branch2c (Conv2D) (None, None, None, 1024)
bn4f_branch2c (BatchNorm) (None, None, None, 1024)
add_13 (Add) (None, None, None, 1024)
res4f_out (Activation) (None, None, None, 1024)
res5a_branch2a (Conv2D) (None, None, None, 512)
bn5a_branch2a (BatchNorm) (None, None, None, 512)
activation_28 (Activation) (None, None, None, 512)
res5a_branch2b (Conv2D) (None, None, None, 512)
bn5a_branch2b (BatchNorm) (None, None, None, 512)
activation_29 (Activation) (None, None, None, 512)
res5a_branch2c (Conv2D) (None, None, None, 2048)
res5a_branch1 (Conv2D) (None, None, None, 2048)
bn5a_branch2c (BatchNorm) (None, None, None, 2048)
bn5a_branch1 (BatchNorm) (None, None, None, 2048)
add_14 (Add) (None, None, None, 2048)
res5a_out (Activation) (None, None, None, 2048)
res5b_branch2a (Conv2D) (None, None, None, 512)
bn5b_branch2a (BatchNorm) (None, None, None, 512)
activation_30 (Activation) (None, None, None, 512)
res5b_branch2b (Conv2D) (None, None, None, 512)
bn5b_branch2b (BatchNorm) (None, None, None, 512)
activation_31 (Activation) (None, None, None, 512)
res5b_branch2c (Conv2D) (None, None, None, 2048)
bn5b_branch2c (BatchNorm) (None, None, None, 2048)
add_15 (Add) (None, None, None, 2048)
res5b_out (Activation) (None, None, None, 2048)
res5c_branch2a (Conv2D) (None, None, None, 512)
bn5c_branch2a (BatchNorm) (None, None, None, 512)
activation_32 (Activation) (None, None, None, 512)
res5c_branch2b (Conv2D) (None, None, None, 512)
bn5c_branch2b (BatchNorm) (None, None, None, 512)
activation_33 (Activation) (None, None, None, 512)
res5c_branch2c (Conv2D) (None, None, None, 2048)
bn5c_branch2c (BatchNorm) (None, None, None, 2048)
add_16 (Add) (None, None, None, 2048)
res5c_out (Activation) (None, None, None, 2048)
fpn_c5p5 (Conv2D) (None, None, None, 256)
fpn_p5upsampled (UpSampling2D) (None, None, None, 256)
fpn_c4p4 (Conv2D) (None, None, None, 256)
fpn_p4add (Add) (None, None, None, 256)
fpn_p4upsampled (UpSampling2D) (None, None, None, 256)
fpn_c3p3 (Conv2D) (None, None, None, 256)
fpn_p3add (Add) (None, None, None, 256)
fpn_p3upsampled (UpSampling2D) (None, None, None, 256)
fpn_c2p2 (Conv2D) (None, None, None, 256)
fpn_p2add (Add) (None, None, None, 256)
fpn_p5 (Conv2D) (None, None, None, 256)
fpn_p2 (Conv2D) (None, None, None, 256)
fpn_p3 (Conv2D) (None, None, None, 256)
fpn_p4 (Conv2D) (None, None, None, 256)
fpn_p6 (MaxPooling2D) (None, None, None, 256)
rpn_model (Model) [(None, None, 2),
(None, None, 2),
(None, None, 4)]
rpn_class (Concatenate) (None, None, 2)
rpn_bbox (Concatenate) (None, None, 4)
input_anchors (InputLayer) (None, None, 4)
ROI (ProposalLayer) (None, 1000, 4)
input_image_meta (InputLayer) (None, 18)
roi_align_classifier (PyramidROIAlign) (None, 1000, 7, 7, 256)
mrcnn_class_conv1 (TimeDistributed) (None, 1000, 1, 1, 1024)
mrcnn_class_bn1 (TimeDistributed) (None, 1000, 1, 1, 1024)
activation_34 (Activation) (None, 1000, 1, 1, 1024)
mrcnn_class_conv2 (TimeDistributed) (None, 1000, 1, 1, 1024)
mrcnn_class_bn2 (TimeDistributed) (None, 1000, 1, 1, 1024)
activation_35 (Activation) (None, 1000, 1, 1, 1024)
pool_squeeze (Lambda) (None, 1000, 1024)
mrcnn_class_logits (TimeDistributed) (None, 1000, 6)
mrcnn_bbox_fc (TimeDistributed) (None, 1000, 24)
mrcnn_class (TimeDistributed) (None, 1000, 6)
mrcnn_bbox (Reshape) (None, 1000, 6, 4)
mrcnn_detection (DetectionLayer) (None, 100, 6)
lambda_3 (Lambda) (None, 100, 4)
roi_align_mask (PyramidROIAlign) (None, 100, 14, 14, 256)
mrcnn_mask_conv1 (TimeDistributed) (None, 100, 14, 14, 256)
mrcnn_mask_bn1 (TimeDistributed) (None, 100, 14, 14, 256)
activation_37 (Activation) (None, 100, 14, 14, 256)
mrcnn_mask_conv2 (TimeDistributed) (None, 100, 14, 14, 256)
mrcnn_mask_bn2 (TimeDistributed) (None, 100, 14, 14, 256)
activation_38 (Activation) (None, 100, 14, 14, 256)
mrcnn_mask_conv3 (TimeDistributed) (None, 100, 14, 14, 256)
mrcnn_mask_bn3 (TimeDistributed) (None, 100, 14, 14, 256)
activation_39 (Activation) (None, 100, 14, 14, 256)
mrcnn_mask_conv4 (TimeDistributed) (None, 100, 14, 14, 256)
mrcnn_mask_bn4 (TimeDistributed) (None, 100, 14, 14, 256)
activation_40 (Activation) (None, 100, 14, 14, 256)
mrcnn_mask_deconv (TimeDistributed) (None, 100, 28, 28, 256)
mrcnn_mask (TimeDistributed) (None, 100, 28, 28, 6)
================================================================
Total params: 44,678,198
Trainable params: 44,618,934
Non-trainable params: 59,264
And my counted Neurons 105,641,486. That looks wrong because there are a lot more than the weights (parameters). I'm not sure if I can really add all layers?
And if anyone's wondering why I want to do this. I want to compare it to a biological neural network, and i have only the neuron count of the brain and not all connections between there. I know they are not comparable but good enough for what I want to do.
thanks for hints and help
A few things:
neurons == filters BatchNormalization layers do have parameters, but I'm not sure you want to consider them as having neurons. Nevertheless, they have learnable parameters for scaling and bias, besides the non-trainable parameters for mean and variance. (A good reason for always using use_bias=False in any layer directly before the batch norm) So, just count the number of filters in each Conv layer. Add the BatchNorm channels if you want to.