I tried to do a meta-analysis of a single proportion. Here is the R codes:
# Packages
library(metafor)
# Data
dat <- dat.debruin2009 #from metafor package
# Metafor package ----
dat <- escalc(measure = "PLO", xi = xi, ni = ni, data = dat)
## Calculate random effect
res <- rma(yi, vi, data = dat)
res
predict(res, transf = transf.ilogit)
Here is the raw result (logit) from res object:
Random-Effects Model (k = 13; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.4014 (SE = 0.1955)
tau (square root of estimated tau^2 value): 0.6336
I^2 (total heterogeneity / total variability): 90.89%
H^2 (total variability / sampling variability): 10.98
Test for Heterogeneity:
Q(df = 12) = 95.9587, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
-0.1121 0.1926 -0.5821 0.5605 -0.4896 0.2654
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
And this is the result from predict():
pred ci.lb ci.ub pi.lb pi.ub
0.4720 0.3800 0.5660 0.1962 0.7660
So, my question is I get a non-significant result from raw result (p = 0.5605). But, a CI from predict() does not crosses zero (CI = 0.3800, 0.5660 ), which indicates a significant result. Do I misunderstand something or missing a step in the R code? or any explanation why the results are contradicting?
===================================================
Edit: I tried using the meta package, I get a similar contradicting result as in metafor.
meta_pkg <- meta::metaprop(xi, ni, data = dat)
meta_pkg$.glmm.random
Here is the result (Similar result as predict() from above):
> meta_pkg
Number of studies combined: k = 13
Number of observations: o = 1516
Number of events: e = 669
proportion 95%-CI
Common effect model 0.4413 [0.4165; 0.4664]
Random effects model 0.4721 [0.3822; 0.5638]
Quantifying heterogeneity:
tau^2 = 0.3787; tau = 0.6154; I^2 = 87.5% [80.4%; 92.0%]; H = 2.83 [2.26; 3.54]
Test of heterogeneity:
Q d.f. p-value Test
95.96 12 < 0.0001 Wald-type
108.77 12 < 0.0001 Likelihood-Ratio
Details on meta-analytical method:
- Random intercept logistic regression model
- Maximum-likelihood estimator for tau^2
- Logit transformation
Similar raw result as in metafor:
> meta_pkg$.glmm.random
Random-Effects Model (k = 13; tau^2 estimator: ML)
tau^2 (estimated amount of total heterogeneity): 0.3787
tau (square root of estimated tau^2 value): 0.6154
I^2 (total heterogeneity / total variability): 90.3989%
H^2 (total variability / sampling variability): 10.4155
Tests for Heterogeneity:
Wld(df = 12) = 95.9587, p-val < .0001
LRT(df = 12) = 108.7653, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
-0.1118 0.1880 -0.5946 0.5521 -0.4804 0.2567
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
The p-value is testing whether the average logit transformed proportion is significantly different from 0. That is not the same as testing whether the proportion is significantly different from 0. In fact, transf.ilogit(0) gives 0.5, so that is the corresponding value of a proportion that is being tested. And you will notice that 0.5 falls inside of the confidence interval after the back-transformation. So everything is fully consistent.