Database of Design Parameters
To support power analyses for single-level and multilevel randomized intervention studies on SEL outcomes, we have generated a comprehensive database that includes (a) point estimates and (b) meta-analytic summaries of design parameters. As discussed in the Data Note, we recommend aligning the design parameters with the key characteristics of the target intervention. This involves considering:
- the target population (total/overall, academic track, non-academic track student population);
- the target experimental design (single-level, two-level, three-level design);
- the target education level (elementary, lower secondary, upper secondary education);
- the target grade (grades 1 to 13);
- the target assessment method of the SEL outcome (student self-report, parent rating, teacher rating);
- the target domain (self-, other-, task-orientation);
- the target context (school-related subject-specific/whole-school academic, general); and
- the target measure.
Importantly, if the target measure or other characteristics of the planned study (e.g., the target grade) do not align well with the characteristics of the samples used to estimate the current design parameters, or if the available point estimates have large standard errors, we recommend utilizing the meta-analytic summaries of the design parameters that most closely reflect the target intervention.
To select the design parameters or their meta-analytic summaries from the tables below, researchers can use the interactive filter functions or adapt the R code provided in OSM B.
Point Estimates
Meta-Analytic Summaries
Table Notes
Show notes
| Key characteristics | ||
|---|---|---|
| Population | Population | Total = Total student population (overall; all school types) |
| Academic = Academic track student population ("Gymnasium") | ||
| Non-academic = Non-academic track student population (other school types) | ||
| Design | Experimental design | 1L-D = Single-level design (individual students independently sampled) |
| 2L-D = Two-level design (students within schools) | ||
| 3L-D = Three-level design (students within classrooms within schools) | ||
| Edu.level | Education level | Elementary = Elementary education (grades 1–4) |
| Lower secondary = Lower secondary education (grades 5–10) | ||
| Upper secondary = Upper secondary education (grades 11–13) | ||
| Method | Assessment method | Student = Student self-report |
| Parent = Parent rating | ||
| Teacher = Teacher rating | ||
| Domain | SEL domain | Self = Self-orientation |
| Other = Other-orientation | ||
| Task = Task-orientation | ||
| Context | SEL context | Mathematics/Science/ICT/Verbal: German/English = School-related: subject-specific |
| School = School-related: whole-school academic | ||
| General = General, context-independent | ||
| Measure | Outcome measure | Several, depends on the context |
| Grade | Grade | Grades 1 to 13 |
| Type | Type of meta-analysis | MA-1 = Across domains and contexts |
| MA-2 = Within domains, across contexts | ||
| MA-3 = Within domains and contexts | ||
| Set | Model set | 0 = No covariate adjustment ("empty") |
| 1-SD = Adjusted for sociodemographic characteristics | ||
| 2-BL = Adjusted for baseline measure of the outcome | ||
| 3-SD+BL = Adjusted for both | ||
| Key statistics | ||
| Parameter | Design parameter | icc_l2 = Intraclass correlation at classroom level (ρClassroom) |
| icc_l3 = Intraclass correlation at school level (ρSchool) | ||
| r2_l1 = Explained variance at student level (R²Student) | ||
| r2_l2 = Explained variance at classroom level (R²Classroom) | ||
| r2_l3 = Explained variance at school level (R²School) | ||
| r2_t = Total explained variance across all students (R²Total; 1L-D only) | ||
| Estimate | Point estimate | |
| Average | Meta-analytic average | |
| SE | Standard error | |
| Further information | ||
| Sample | Probability sample | DESI = Assessment of Student Achievements in German and English as a Foreign Language |
| IQB Trends = IQB Trends in Student Achievement 2016 | ||
| PISA 2003 = Programme for International Student Assessment, 2003 cycle | ||
| PISA 2006 = Programme for International Student Assessment, 2006 cycle | ||
| PISA 2009 = Programme for International Student Assessment, 2009 cycle | ||
| PISA 2012 = Programme for International Student Assessment, 2012 cycle | ||
| NEPS-SC2 = National Educational Panel Study, starting cohort 2—kindergarten | ||
| NEPS-SC3 = National Educational Panel Study, starting cohort 3—grade 5 | ||
| NEPS-SC4 = National Educational Panel Study, starting cohort 4—grade 9 | ||
| Wave | Assessment wave | |
| Lag | Time lag (months) | |
| Variable | Outcome variable | |
| Reliability | Outcome reliability | |
| n_Students | Number of students | |
| n_Classrooms | Number of classrooms | |
| n_Schools | Number of schools | |
| n_Covariates | Number of covariates | |
| k | Observed design parameters | |
| J | Number of samples | |
| Fit.method | Fitting method | FE = Fixed-effect model |
| REML = Random-effects model | ||
| CI.lb | 95% Confidence interval: lower bound | |
| CI.ub | 95% Confidence interval: upper bound | |
| PI.lb | Prediction interval: lower bound | (REML only) |
| PI.ub | Prediction interval: upper bound | (REML only) |
| Tau2_Within | Within-sample heterogeneity | (τ²Within; REML only) |
| Tau2_Between | Between-sample heterogeneity | (τ²Between; REML only) |
| I2_Total | I² statistics: total | (I²Total; REML only) |
| I2_Within | I² statistics: within samples | (I²Within; REML only) |
| I2_Between | I² statistics: between samples | (I²Between; REML only) |