Search code examples
rstatisticsexperimental-design

How do you run a complier average causal effect (CACE) analysis in R?

Context

When conducting a Randomized-Controlled Trial (RCT), some participants are randomly assigned to a treatment condition, and others to a control group. However, not everyone assigned to the treatment might follow the treatment protocol (called "treatment compliance").

According to Sagarin et al. (2014), one sensible approach to address this problem is using the complier average causal effect (CACE), also sometimes known as Local average treatment effect (LATE). According to Wikipedia, it is "the treatment effect for the subset of the sample that takes the treatment if and only if they were assigned to the treatment, otherwise known as the compliers." In other words, it will be useful if a proportion of your participants assigned to the treatment group did not follow the treatment protocol.

Question

How do you run this analysis in R?

I couldn't find anything precise on this from Google and stackoverflow searches.

Also despite my many readings, I still cannot figure out what the expected outcome is supposed to be. When using CACE, what is the outcome? Do you end up with updated scores/data ajusted for treatment non-compliance that you can just plug in your regular analyses (akin to factor scores)? Or do you simply get some kind of number that you have to do something with?

What I've tried

The eefAnalytics package seems to provide the most convenient function for this: caceSRTBoot(). "caceSRTBoot performs exploraty CACE analysis of simple randomised education trials." It allows to specify compliance through a simple compliance percentage (beautifully simple and convenient).

However, I am experiencing some problems installing the eefAnalytics package while trying to test it to see the kind of output it gives:

install.packages("eefAnalytics")
package ‘eefAnalytics’ is not available (for R version 4.0.2)

# Install the latest version of this package by entering the following in R:
install.packages("eefAnalytics", repos="http://R-Forge.R-project.org")
package ‘eefAnalytics’ is not available (for R version 4.0.2)
Warning in install.packages :
  unable to access index for repository http://R-Forge.R-project.org/bin/windows/contrib/4.0:
  cannot open URL 'http://R-Forge.R-project.org/bin/windows/contrib/4.0/PACKAGES'

Upon closer investigation, Cran says: "Package ‘eefAnalytics’ was removed from the CRAN repository."

The other packages I've looked at (e.g., 1, 2, 3, 4) seemed quite complicated and I couldn't figure them out (they don't have a parameter for % compliance for instance, and I had trouble making their "Run this example" widget work). Is there any other user-friendly package out there? Is there any other way to do this analysis in R? Would anyone have some kind of "tutorial"?

Relevant pages: 1.

Solution

  • Installing the eefAnalytics package

    I contacted the eefAnalytics package maintainer through the package documentation. I was told that an updated version will be available soon in R. In the meanwhile, I was able to install the old version of the package from the CRAN archives with:

    install.packages("https://cran.r-project.org/src/contrib/Archive/eefAnalytics/eefAnalytics_1.0.6.tar.gz", repos = NULL, type = "source")

    Note, however, that I had to manually install packages geoR and metafor separately first (else it was throwing an error).

    Running the Causal Average Treatment Effect

    Answering the first part of the question:

    How do you run this analysis in R?

    Running the example available from documentation for a simple randomised trial, we get:

    library(eefAnalytics)    
data(mstData)
############# weighted ITT ####################################
caceOutput3 <- caceSRTBoot(Posttest~ Prettest+ Intervention,
               intervention="Intervention",
               compliance = "Percentage_Attendance",nBoot=1000,data=mstData)

cace <- caceOutput3$CACE
cace
  Compliance   ES   LB   UB
1       P> 0 0.32 0.04 0.62
2      P> 10 0.32 0.04 0.62
3      P> 20 0.37 0.04 0.72
4      P> 30 0.42 0.05 0.83
5      P> 40 0.47 0.06 0.92
6      P> 50 0.58 0.07 1.18

Complier <- caceOutput3$Compliers
Complier    
        P > 0 P > 10 P > 20 P > 30 P > 40 P > 50 P > 60 P > 70 P > 80 P > 90
pT          1      1   0.87   0.75   0.69   0.55   0.41   0.31   0.25   0.15
pC          0      0   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00
P=PT-pC     1      1   0.87   0.75   0.69   0.55   0.41   0.31   0.25   0.15

### visualising CACE effect size

plot(caceOutput3)

    enter image description here

    Interpretation of output

    I couldn’t find this information from the package documentation, but from what I understand from the output:

    ITT = intent to treat

    ES = effect size (Hedge’s g)

    LB = lower-bound (of the confidence interval of the effect size)

    UB = upper-bound

    pT = Percentage of compliers in Treatment group

    pC = Percentage of compliers in Control group

    P=PT-pC = Percentage of compliers in Treatment group minus Percentage of compliers Control group

    P > X = Value for participants with a percentage of compliance greater than X (e.g., 50%)

    What is the outcome?

    To answer the second part of the question:

    When using CACE, what is the outcome?

    The main outcome of interest seems to be an adjusted effect size (Hedge's g, which is similar to Cohen's d but better for small sample sizes < 20). It seems only possible to compare two groups together, not more (as Hedge's g, like Cohen's d, can only compare two means at once).

    The plot is very useful and allows to see the "improvement" of the effect size as a function of increasing compliance with treatment. What you see in this example is that a higher compliance percentage leads to larger effect sizes, as expected.