Search code examples
rmodelmetafor

How can I^2 ==0 (heterogeneity measure) in RMA random effects meta analysis in R?

I'm working on a meta-analysis of epidemiological studies. The studies are very heterogeneous in terms of population, intervention and analysis, so I'm using a random effects model for meta-analysis using "metafor" in R.

I subsetted the studies into subgroups with comparable outcomes. 5/6 look fine.

However, there is one subgroup that looks all wrong because tau is 0 and I^2 is 0. Looking at the data, I don't see why total heterogeneity would be 0. 

res <- rma(yi=beta, sei=se, slab=(1:7), measure="OR",data=SIPVdata,  digits=3, method= "ML")

Random-Effects Model (k = 3; tau^2 estimator: ML)

  logLik  deviance       AIC       BIC      AICc  
  -0.217     2.635     4.433     2.630    16.433  

tau^2 (estimated amount of total heterogeneity): 0.000 (SE = 0.044)
tau (square root of estimated tau^2 value):      0.001
I^2 (total heterogeneity / total variability):   0.00%
H^2 (total variability / sampling variability):  1.00

Test for Heterogeneity: 
Q(df = 2) = 2.635, p-val = 0.268

Model Results:

estimate     se    zval   pval   ci.lb   ci.ub   
  -0.350  0.145  -2.417  0.016  -0.634  -0.066  *

Plotting the model output looks like this:

img

So you can see that 2 observations (5&3), which have small confidence intervals and similar estimates, have the most influence in the sample. The other estimates have wide CIs, which all overlap. I might expect the estimate heterogeneity to be low in this case, but not 0, and certainly not the total variability tau.

Does anyone have an idea what is going on in this meta-analysis?

Thank you very much!

Solution

  • The ML estimator of tau^2 is known to have negative bias. That of course does not mean that it is too low in this particular case, but I would suggest to switch to an estimator that is known to be approximately unbiased. The one I would recommend is REML. This is in fact the default estimator (i.e., if you do not specify the method argument).

    Also, note that with 7 studies, the estimate of tau^2 (and hence I^2) is not going to be very precise. Run confint(res) and you will see that the confidence interval for I^2 is going to be very wide. In other words, all values within the CI are compatible with these data, so really there could indeed be no heterogeneity or a lot of it.