I'd like to standardize my data to zero mean and std = 1. The shape of my data is 28783x4x24x7, and it can thought of as 28783 images with 4 channels and dimensions 24x7. The channels need to be standardized. How do I standardize while specifying that the 2nd dimension holds the features?
I found a way to do it. It's perhaps not the most efficient, but it also allows me to use this approach for cross-validation, where I only want to obtain the mean and std from my training data, but apply the operation to both training and test data. This can be used across any number of dimensions if you only want the mean of one dimension. See example code below:
n_user = 3
n_channel = 2
n_pixels = 3
A = np.zeros(shape=(n_user, n_channel, n_pixels))
for i in range(n_user):
A[i, 0, :] = np.arange(n_pixels)
A[i, 1, :] = np.arange(n_pixels) + n_pixels
print A
mu_f = np.zeros(shape=n_channel)
sigma_f = np.zeros(shape=n_channel)
for i in range(n_channel):
mu_f[i] = np.mean(A[:,i,:])
sigma_f[i] = np.std(A[:,i,:])
print mu_f
print sigma_f
for i in range(n_channel):
A[:, i, :] -= mu_f[i]
A[:, i, :] /= sigma_f[i]
print A
print np.mean(A[:,0,:])
print np.std(A[:,0,:])