Imagine that you have a deep neural network on a regression problem to predict weight of a person. Your neural network looks like this:
Dense(112, activation='relu')
Dense(512, activation='relu)
Dropout(0.5)
Dense(1, activation='relu')
Now during training, let's say that 50 % of nodes corresponds to an output of around 30 to 100 depending on the input. Now during testing, when the dropout is not happening, will not the output of around 30 to 100 double because previously, when just 50 % of nodes were active, this 50 % of nodes were passing some value to the output node, and thus we were getting output of around 30 to 100, and now during testing, when all the nodes are active, so all the nodes are passing some value to the output node. So if 50 % of nodes got an output of around 30 to 100 will not this value be doubled during testing, when 100 % nodes are active?
As even said by Dr. Snoopy, in the comments, that dropout has a functionality (multiplication by p), to prevent the problem I am asking about.
If a unit is retained by a probability of p, then at test time, all the ongoing weights of that unit will be first multiplied by p.
So, in the case of predicting weights that i mentioned in the question, all the ongoing weights after the dropout layer will first be multiplied by p (0.5 in this case), making the test output same as training output, of around 30 to 100, and fixing the problem!