HMMlearn GMMHMM error

Question

I am trying to initialize several GMM's for use with the GMMHMM's gmms_ attribute. Each GMM instance has a different mean, weight and co-variance and serves as a component of a 5-component mixture for the GMMHMM. The mean, weight and co-variance are determined from a (5-cluster) k-means algorithm of the data-set I want to fit, where the mean is center of each cluster, the weight is the weight of each cluster and the co-variance is the - you guessed it - co-variance of each cluster.

Here is a code snippet:

X_clusters = cls.KMeans(n_clusters=5)
fitted_X = X_clusters.fit(X)
means = fitted_X.cluster_centers_
cluster_arrays = extract_feat(X, fitted_X.labels_)
print ('Means: {0}'.format(means))

total_cluster = float(len(X)) 
all_GMM_params = []
for cluster in cluster_arrays:
    GMM_params = []
    weight = float(len(cluster))/total_cluster
    covar = np.cov(cluster)
    GMM_params.append(weight)
    GMM_params.append(covar)
    all_GMM_params.append(GMM_params)

for i in range(len(means)):
    all_GMM_params[i].append(means[i])


model = GMMHMM(n_components=4, covariance_type="diag", n_iter=1000,
            n_mix = 5, algorithm='map')

for i in range(len(all_GMM_params)):
    GMM_n = mix.GMM(init_params = '')
    GMM_n.weights_ = np.array(all_GMM_params[i][0])
    GMM_n.covars_ = np.array(all_GMM_params[i][1])
    GMM_n.means_ = np.array(all_GMM_params[i][2])
    model.gmms_.append(GMM_n)

model.fit(X)

When I try to fit the model, however, I get the following error:

fitting to HMM and decoding ...Traceback (most recent call last):
  File "HMM_stock_sim.py", line 156, in <module>
    model.fit(X)
  File "C:\Python27\lib\site-packages\hmmlearn\base.py", line 436, in fit
    bwdlattice)
  File "C:\Python27\lib\site-packages\hmmlearn\hmm.py", line 590, in _accumulate
_sufficient_statistics
    stats, X, framelogprob, posteriors, fwdlattice, bwdlattice)
  File "C:\Python27\lib\site-packages\hmmlearn\base.py", line 614, in _accumulat
e_sufficient_statistics
    stats['start'] += posteriors[0]
ValueError: operands could not be broadcast together with shapes (4,) (9,) (4,)

I have never seen error like this before, its my first time working with sklearn and HMMlearn. How do I go about fixing this error?

Answer 1

I was able to reproduce the issue using a random sample from a two-component Gaussian mixture:

import numpy as np

X = np.append(np.random.normal(0, size=1024),
              np.random.normal(4, size=1024))[:, np.newaxis]

So here's my take on why your code doesn't work. np.cov treats each row of a given array as a variable. Thus for an array of shape (N, 1) the output is bound to be of shape (N, N). Clearly, this isn't what you want, since the covariance matrix for a 1-D Gaussian is simply a scalar.

The solution is to transpose cluster before passing it to np.cov:

np.cov(cluster.T)    # has shape () aka scalar

---

After switching to a 3-D X I've spotted two more issues:

• n_mix is the number of components in a GMM, while n_components refers to the number of Markov chain states (or equivalently the number of mixtures). Note that you pass n_components=4 to the GMMHMM constructor and then append 5 GMM instances to model.gmms_.
• Moreover, GMMHMM pre-populates model.gmms_ so you end up with n_components + 5 instead of 4 mixtures which explains the (9, ) mismatch.

Updated code:

#      the updated parameter value.
#              vvvvvvvvvvvvvv
model = GMMHMM(n_components=5, covariance_type="diag", n_iter=1000,
               n_mix=5, algorithm='map')
#              ^^^^^^^
#  doesn't have to match n_components

for i, GMM_n in enumerate(model.gmms_):
    GMM_n.weights_ = ...
    # Change the attributes of an existing instance 
    # instead of appending a new one to ``model.gmms_``.