Workshop: Power Analysis for Experimental Designs in R with the PowerUpR Package

Author

Sophie E. Stallasch | University of Potsdam

Welcome to the introductory workshop on power analysis for single- and multilevel randomized trials (RTs) in R at the “Nachwuchstagung” of the GEBF 2024 at the University of Potsdam, taking place on March 21, 2024!

This online tutorial illustrates diverse application scenarios and offers corresponding exercises to perform power analysis with the PowerUpR package (Bulus et al., 2021). In this two-part workshop, you learn the basic steps to follow when designing…

PART I: Single-level designs … simple individually randomized trials (IRTs)
PART II: Multilevel designs … more complex two- and three-level cluster randomized trials (CRTs)

Setup

First of all, check that you have the latest version of R and RStudio.

You can install the latest version of PowerUpR (currently, this is version 1.1.0) from CRAN. Throughout this tutorial, we will also use a bunch of tidyverse (Wickham et al., 2019) functions, as well as the kableExtra package (Zhu, 2024), so make sure you have installed these packages (for details on the versions used here, see the section “Session Info” below).

# -- required packages
pkg <- c("PowerUpR", "tidyverse", "kableExtra")


# -- load packages (or install from CRAN if not installed)
invisible(
  lapply(pkg,
         function(x) {
           if (!require(x, character.only = TRUE)) {
             install.packages(x, dependencies = TRUE)
             require(x, character.only = TRUE)}
         }
  )
)

The PowerUpR Package in a Nutshell

PowerUpR is the R implementation of the ‘PowerUp!’ Excel tool created by Dong & Maynard (2013), which is accessible at the ‘Causal Evaluation’ website. PowerUpR provides functions to conduct three types of power analysis, depending on the desired output and the applied design. Specifically, PowerUpR offers power analysis tools to plan individually, multisite, and cluster randomized designs with up to four hierarchical levels. Note that this workshop does not cover multisite trials (MSRTs). However, if you want to learn about power analysis for such designs, you may want to check out my tutorials at https://sophiestallasch.github.io/2022-workshop-CRT/#Multisite_Trials).

Type of power analysis: Output	IRT functions	2L-CRT functions	3L-CRT functions
Minimum required sample size (MRSS)	`mrss.ira()`	`mrss.cra2()`	`mrss.cra3()`
Minimum detectable effect size (MDES)	`mdes.ira()`	`mdes.cra2()`	`mdes.cra3()`
Power	`power.ira()`	`power.cra()`	`power.cra3()`

Go to the PowerUpR Documentation to see details on the arguments etc.

Tip

At the ‘Causal Evaluation’ website, you also find the PowerUpR Shiny App, which nicely implements (and visualizes) most of the functionality of the PowerUpR package.

Worked Example for the Workshop

To illustrate the process of performing power analysis with PowerUpR, we will use the following worked example of an intervention to foster student achievement.

Worked Example

A research team has programmed a comprehensive learning app. It is meant to function as a multidisciplinary digital learning environment for students in various grade levels in the German school system. Therefore, it can be used to support teaching and learning in manifold achievement domains throughout the entire school career. The researchers aim to experimentally evaluate whether the app actually improves student achievement.

Session Info

R version 4.3.3 (2024-02-29 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 11 x64 (build 22631)

Matrix products: default


locale:
[1] LC_COLLATE=German_Germany.utf8  LC_CTYPE=German_Germany.utf8   
[3] LC_MONETARY=German_Germany.utf8 LC_NUMERIC=C                   
[5] LC_TIME=German_Germany.utf8    

time zone: Europe/Berlin
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] kableExtra_1.4.0 lubridate_1.9.3  forcats_1.0.0    stringr_1.5.1   
 [5] dplyr_1.1.4      purrr_1.0.2      readr_2.1.5      tidyr_1.3.1     
 [9] tibble_3.2.1     ggplot2_3.5.0    tidyverse_2.0.0  PowerUpR_1.1.0  

loaded via a namespace (and not attached):
 [1] gtable_0.3.4      jsonlite_1.8.8    compiler_4.3.3    tidyselect_1.2.0 
 [5] xml2_1.3.6        systemfonts_1.0.5 scales_1.3.0      yaml_2.3.8       
 [9] fastmap_1.1.1     R6_2.5.1          generics_0.1.3    knitr_1.45       
[13] htmlwidgets_1.6.4 munsell_0.5.0     svglite_2.1.3     pillar_1.9.0     
[17] tzdb_0.4.0        rlang_1.1.3       utf8_1.2.4        stringi_1.8.3    
[21] xfun_0.42         viridisLite_0.4.2 timechange_0.3.0  cli_3.6.2        
[25] withr_3.0.0       magrittr_2.0.3    digest_0.6.34     grid_4.3.3       
[29] rstudioapi_0.15.0 hms_1.1.3         lifecycle_1.0.4   vctrs_0.6.5      
[33] evaluate_0.23     glue_1.7.0        fansi_1.0.6       colorspace_2.1-0 
[37] rmarkdown_2.26    tools_4.3.3       pkgconfig_2.0.3   htmltools_0.5.7

References

Bulus, M., Dong, N., Kelcey, B., & Spybrook, J. (2021). PowerUpR: Power analysis tools for multilevel randomized experiments. https://CRAN.R-project.org/package=PowerUpR

Dong, N., & Maynard, R. (2013). PowerUp!: A Tool for Calculating Minimum Detectable Effect Sizes and Minimum Required Sample Sizes for Experimental and Quasi-Experimental Design Studies. Journal of Research on Educational Effectiveness, 6(1), 24–67. https://doi.org/10.1080/19345747.2012.673143

Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L. D., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., Kuhn, M., Pedersen, T. L., Miller, E., Bache, S. M., Müller, K., Ooms, J., Robinson, D., Seidel, D. P., Spinu, V., … Yutani, H. (2019). Welcome to the tidyverse. 4, 1686. https://doi.org/10.21105/joss.01686

Zhu, H. (2024). kableExtra: Construct complex table with ’kable’ and pipe syntax. https://CRAN.R-project.org/package=kableExtra