I am trying hard to get convergence in Baum-Welch algorithm but could not able to do this because of Log-likelihood, AIC, BIC is showing NaN. My data contain only one column. Please help me in this regard i will be very thankful to you. The detailed codes are given below
library(RHmm)
obs<-read.table("C:\\Users\\Administrator\\Desktop\\SPI-TESTS.csv",header=T,sep=",")
hm_model <- HMMFit(obs=obs, dis="MIXTURE", nStates=11, nMixt=4, control=list(iter=2000))
print (hm_model)
Call:
----
HMMFit(obs = obs, dis = "MIXTURE", nStates = 11, nMixt = 4, control = list(iter = 2000))
Model:
------
11 states HMM with mixture of 4 gaussian distribution
Baum-Welch algorithm status:
----------------------------
NO CONVERGENCE AFTER 265 ITERATIONS
PROBLEM IN BAUM-WELCH'S ALGORITHM
Last Estimation:
----------------
Initial probabilities:
Pi 1 Pi 2 Pi 3 Pi 4 Pi 5 Pi 6 Pi 7 Pi 8 Pi 9 Pi 10 Pi 11
0 0 0 0 0 0 0 0 3.779559e-232 2.509294e-79 1
Transition matrix:
State 1 State 2 State 3 State 4 State 5 State 6 State 7 State 8 State 9 State 10 State 11
State 1 1.158892e-12 5.513564e-01 1.406934e-01 2.653148e-01 2.769269e-02 1.553028e-37 1.494273e-02 9.800663e-13 1.507891e-49 4.715263e-34 4.017981e-18
State 2 2.654186e-02 4.087015e-23 3.204037e-10 2.212773e-01 5.021474e-02 4.005844e-20 4.165265e-12 5.073141e-01 1.946520e-01 1.632596e-32 1.113244e-71
State 3 3.241748e-10 1.016694e-40 3.788610e-02 3.320507e-19 5.677861e-39 4.381774e-01 3.887688e-25 6.912725e-19 1.182014e-45 1.517400e-35 5.239365e-01
State 4 5.644328e-17 3.149625e-51 1.875990e-37 7.768767e-02 9.337121e-02 3.695106e-02 4.546543e-31 3.693171e-01 1.411264e-56 4.226730e-01 2.787318e-68
State 5 2.703254e-17 1.232838e-22 3.593962e-45 6.150321e-02 3.976687e-29 2.572677e-24 9.384968e-01 7.366599e-09 2.384907e-33 5.458220e-27 1.044629e-10
State 6 1.927068e-23 1.142987e-31 6.133354e-11 4.227546e-37 6.058461e-19 3.366786e-01 3.878805e-25 1.295066e-31 1.298184e-25 6.633214e-01 3.737511e-35
State 7 8.304118e-01 3.099592e-17 1.201420e-01 3.844308e-48 5.197072e-22 1.869553e-22 4.401821e-33 4.944619e-02 8.966040e-38 7.440181e-17 2.103199e-77
State 8 2.608467e-23 2.128828e-01 3.623140e-01 6.605205e-02 4.193308e-26 7.099826e-02 8.209803e-15 1.495140e-26 2.877529e-01 2.419725e-31 6.329163e-25
State 9 6.321264e-18 1.020369e-21 2.233158e-97 1.150197e-20 3.578052e-17 1.140077e-56 9.255432e-01 3.718692e-23 8.778677e-69 1.698017e-74 7.445679e-02
State 10 1.166198e-35 2.422477e-17 2.629801e-33 6.517930e-16 4.811069e-01 9.681698e-07 3.583564e-17 6.114747e-26 1.577586e-01 1.619016e-27 3.611335e-01
State 11 1.476885e-01 5.291609e-44 4.304542e-02 5.421176e-48 2.279369e-01 4.006380e-21 1.154627e-20 1.283353e-50 7.747702e-02 3.628620e-01 1.409901e-01
Conditional distribution parameters:
Distribution parameters:
State 1
mean var prop
mixt. 1 0.3228887 0.0670044730 0.01175632
mixt. 2 2.2903418 0.0004215949 0.02883315
mixt. 3 0.6950723 0.1498205389 0.39625490
mixt. 4 0.8153021 0.1616678607 0.56315563
State 2
mean var prop
mixt. 1 0.9000000 -9.768662e-17 0.04267843
mixt. 2 1.0115568 1.583260e-03 0.06245711
mixt. 3 1.3279109 8.353099e-03 0.16728413
mixt. 4 0.3209322 2.318155e-02 0.72758033
State 3
mean var prop
mixt. 1 -0.01040615 0.0001118124 0.05114568
mixt. 2 1.28399423 0.0007558368 0.19161416
mixt. 3 -0.51378918 0.0012591050 0.33529535
mixt. 4 -0.26900326 0.0114199682 0.42194481
State 4
mean var prop
mixt. 1 -0.5196946 0.0370986863 0.06308098
mixt. 2 -0.4491540 0.0001833145 0.11588290
mixt. 3 -0.3999082 0.0194713200 0.17269967
mixt. 4 -0.8223137 0.0155369698 0.64833645
State 5
mean var prop
mixt. 1 -0.54558081 0.007469851 0.07947785
mixt. 2 2.29433769 0.162783811 0.16413279
mixt. 3 -0.46162548 0.002112128 0.17196909
mixt. 4 -0.01918344 0.011286404 0.58442027
State 6
mean var prop
mixt. 1 0.04382151 0.004894925 0.09060242
mixt. 2 0.78022790 0.146280792 0.24418142
mixt. 3 1.67034964 0.007670877 0.25471853
mixt. 4 -0.12843702 0.003420775 0.41049763
State 7
mean var prop
mixt. 1 -0.5438470 0.06310705 0.1163411
mixt. 2 -0.4574091 0.07136046 0.2189112
mixt. 3 -0.5405554 0.06353600 0.2381260
mixt. 4 0.5991808 0.10922573 0.4266217
State 8
mean var prop
mixt. 1 0.35036456 0.01848146 0.1217106
mixt. 2 0.32292322 0.01495311 0.1712873
mixt. 3 0.06647903 0.01427462 0.1834535
mixt. 4 0.68608747 0.02160377 0.5235487
State 9
mean var prop
mixt. 1 -1.401972 0.023920717 0.02111203
mixt. 2 -1.737035 0.002683553 0.26584908
mixt. 3 -1.175300 0.021111569 0.35446196
mixt. 4 -2.424672 0.319739380 0.35857693
State 10
mean var prop
mixt. 1 -2.0511859 0.01757202 0.09649034
mixt. 2 -0.7537656 0.03633816 0.14177569
mixt. 3 0.2397362 0.14818220 0.30284293
mixt. 4 1.4524058 0.19584007 0.45889105
State 11
mean var prop
mixt. 1 -1.002009 0.004011447 0.06652293
mixt. 2 -1.007930 0.004125391 0.11683122
mixt. 3 -1.122122 0.001598325 0.29372147
mixt. 4 -1.497583 0.010445953 0.52292439
Log-likelihood: NaN
BIC criterium: NaN
AIC criterium: NaN
0.11
-0.9
-1.67
1.13
0.39
1.25
-1.48
-1.42
0.67
-1.41
0.36
-0.42
0.91
0.56
0.39
0.45
-0.17
0.68
-1.01
1.84
0.17
1.28
0.63
0.42
-2.97
0.38
1.04
0.98
0.51
-2.05
-0.7
0.14
-0.33
-1.14
0.83
-0.9
-0.12
1.41
0
-0.21
0.32
0.11
0.23
0
-1.03
-0.47
0.36
0.95
-0.7
-2.07
-1.16
-0.72
-1.25
-0.73
0.23
-0.9
-0.04
-1.8
-0.49
0.46
-0.7
0.67
-0.02
-1.41
0.23
0.05
0.95
0.85
1.34
-0.11
-0.52
-1.09
-0.12
0.54
-0.38
-0.96
0.17
-1.1
1.61
0.47
-2.44
-0.86
0.74
0.29
-0.34
0.47
-0.08
1.77
1.92
-0.22
-0.42
0.61
-0.53
1.57
-0.7
-1.75
0.71
0.9
0.4
0.21
1.07
0.12
1.25
-0.07
0.62
-0.61
-0.12
0.33
1.57
1.31
1.63
-0.15
0.63
-1.1
-1.01
-1.48
0.84
1.47
-0.96
-0.62
0.79
1.71
-0.38
0.86
0.12
-0.78
-0.4
-2.4
-0.6
1.47
-0.7
1.54
-1.74
0.34
1.37
-0.48
0.94
-0.2
-0.11
-0.03
2.16
0.04
0.24
-0.48
1.7
0.97
-1.08
-0.43
-0.95
0.16
-1.52
-0.99
-1.06
-1.9
-0.24
-0.99
0.67
-0.86
0.33
-3.6
-1.14
1.37
-1.5
0.91
0.42
-0.75
-0.81
0.07
-0.57
0.96
0.29
-0.26
0.26
-0.52
-1.19
-1.42
-0.54
-1.14
0.15
-0.2
1.2
1.26
0.06
-0.49
-0.15
1.61
2.6
-0.18
1.27
-0.45
-1.09
0.11
-0.42
1.27
-1.64
-0.04
-1.7
0.07
-0.72
0.63
0.34
0.27
-1.23
-0.99
0.92
-0.26
-0.55
1.68
1.17
1.83
1.11
-0.44
-0.25
0.45
0.37
1
0.22
0.49
-0.45
0.74
-1.1
-0.99
-0.37
0.92
-0.26
0.02
-0.18
0.07
-0.99
-0.16
-0.07
-0.17
1.6
1.08
0.08
-0.49
0.74
2.35
-0.37
-0.42
-1.46
-2.83
0.24
-0.59
-1.19
1.48
-0.04
-0.85
1.21
0.18
0.66
-1.47
-0.24
0.56
-1.06
0.43
-0.49
-1.61
-0.53
-0.41
1.41
-0.47
-1.97
-0.65
-0.21
-0.13
-1.13
1.45
-1.42
1.06
-1.34
1.02
0.99
0.11
1.31
0.44
1.67
-1.09
0.7
1.56
1.27
-1.71
0.6
0.03
1.41
-0.41
0.06
2.73
1.25
0.62
0.21
0.66
-2.81
0.51
0.51
-1.79
0.18
0.94
0.46
-0.52
-0.7
-0.32
0.05
-0.38
0.39
0.58
0.81
0.59
-0.91
-0.78
1.53
-0.02
-0.89
0.1
0.24
-0.24
-0.49
-0.78
0.48
0.9
0.87
0.24
-1.23
-1.04
-1.05
-0.63
0.11
0.54
-1.7
-0.3
0.94
-0.24
-1.42
-0.39
-0.44
-2.17
-0.53
-0.63
-0.11
0.29
0.68
-0.15
-1.52
-1.45
-1.61
0.75
-0.67
0.65
-0.28
-0.18
0.13
-0.09
0.71
0.75
-1.04
2.35
-0.64
0.6
1.28
1
0.52
-0.11
0
0.84
0.36
-2.23
0.91
0.34
-0.21
-1.52
-0.28
-1.4
-0.45
-0.37
1.17
-0.54
0.81
-0.78
-0.91
-0.08
-0.66
0.07
-0.9
0.37
-0.34
-1.54
-2.21
-1.75
0.69
0.67
-0.88
-0.08
0.53
0.8
-0.75
2.34
-1.11
-1.71
2.72
0.52
0.75
0.9
0.03
0.23
-1.28
-0.57
2.27
0.31
-0.94
0.61
0.05
0.76
-1.18
-0.06
0.33
1.27
-0.44
2.02
-1.42
-0.05
0.61
0.21
-0.73
0.89
-0.49
0.11
1.37
-1.39
1.14
0.04
0.67
0.37
-0.74
0.32
-1.31
0.88
-0.52
-0.24
-0.02
-0.86
-0.51
1.03
0.47
-0.45
0.63
1.19
0.24
-0.52
1.13
1.33
-1.12
0.59
-0.12
0.56
1.23
1.31
-0.09
0.23
0.16
-1.08
0.99
-0.04
0.39
0.47
0.1
-1.8
-0.75
0.97
1.42
0.34
-0.99
-0.55
0.83
0.53
-0.07
0.03
0.65
0.1
-2.03
0.63
0.73
0.12
1.81
-0.54
-0.36
-1.67
1.29
0.56
0.59
0.35
0.79
-0.58
0.77
-1.62
-0.6
0.39
0.2
0.64
1.58
1.51
-0.01
-0.05
2.31
-0.9
1.5
-1.54
2.08
-1.3
-0.66
0.52
0.42
0.8
-2.44
-0.34
1.03
0.41
I read somewhere that RHmm is outdated. The fact that it is no longer mantained makes me believe that that NaN result is a bug that wasn't fixed or even documented. Wherever I read that (I think it was here at StackOverflow, but couldn't find the post again), they recommended using the package depmixS4 instead of RHmm.
If you want to continue using RHmm, I've come up with a few workarounds. Increasing solely the max. number of iterations proved inneffective, since the algorithm crashes most of the times before even hitting the default 500 iterations.
First, you might want to loop until convergence. Doing that will increase the overall runtime of the script by a a huge amount, especially if you are re-fitting the model over and over again. To counter that, I increased the tolerance ('tol' argument of the control parameter) of the relative log-likehood augmentation to 1e-3.
The default value is 1e-6. My guess is that the Baum-Welch implementation is underflowing at some point when trying to reach that tolerance. Increasing it to 1e-3 will make the algorithm consider a convergence when the last increment of the LLH is < 0.001, instead of 0.000001.
This will of course decrease the LLH (the precision) of your model and increase the overall error, but makes it time-feasible.
The resulting code might look something like:
baum_welch <- function(observed, max_iter, tolerance) {
i <- 1
repeat {
hm_model <- HMMFit(obs = observed, nStates = 5, nMixt = 4, dis = "MIXTURE", control = list(iter=max_iter , tol=tolerance))
if (hm_model$convergence) {
cat("Tries: ", i, "\n")
break
}
i <- i + 1
}
hm_model
}
baum_welch(train_set, 2000, 1e-3)