There have three common image data normalization methods, which are
1. X = (X - X.mean) / X.std
2. X /= 255. # (based on formula: (X - min) / (max - min) which can converge data into [0, 1].)
3. X = 2 * (X - min) / (max - min) - 1 # converge into [-1, 1]
I found in different CNN tutorials or posts, people may use one of them to normalize data. But I am a bit confused about them, how should I select one in different situations?
Thanks for any explanations in advance.
Broadly speaking, the reason we normalize the images is to make the model converge faster. When the data is not normalized, the shared weights of the network have different calibrations for different features, which can make the cost function to converge very slowly and ineffectively. Normalizing the data makes the cost function much easier to train.
Exactly which normalization method you choose depends on the data that you are dealing with and the assumptions you make about that data. All the above three normalization methods are based on two ideas, that are, centering and scaling. Method 2. involves only scaling the data into a particular range. This makes sure that the scale of the various features is in a similar range and hence gives stable gradients. Method 1. involves centering the data around the mean datapoint and then dividing each dimension of the datapoint with its standard deviation so that all the dimensions hold equal importance for the learning algorithm. This normalization is more effective when you have a reason to believe that different dimensions in the data have vastly different range. Bringing all the dimensions in the same range thus make sharing of the parameters effective. Method 3 can also be seen as somewaht doing the sam job as method 1.