Effects of intense assessment on statistical power in randomized controlled trials: Simulation study on depression.

Schuster, Raphael; Schreyer, Manuela Larissa; Kaiser, Tim; Berger, Thomas; Klein, Jan Philipp; Moritz, Steffen; Laireiter, Anton-Rupert; Trutschnig, Wolfgang (2020). Effects of intense assessment on statistical power in randomized controlled trials: Simulation study on depression. Internet Interventions, 20, p. 100313. Elsevier 10.1016/j.invent.2020.100313

Preview	Text Effects of intense assessment on statistical power in randomized controlled trials_ Informed simulation study on depression.pdf - Accepted Version Available under License Creative Commons: Attribution-Noncommercial-No Derivative Works (CC-BY-NC-ND). Download (1MB) | Preview
Preview	Text 1-s2.0-S221478291930082X-main.pdf - Published Version Available under License Creative Commons: Attribution-Noncommercial-No Derivative Works (CC-BY-NC-ND). Download (1MB) | Preview

Smartphone-based devices are increasingly recognized to assess disease symptoms in daily life (e.g. ecological momentary assessment, EMA). Despite this development in digital psychiatry, clinical trials are mainly based on point assessments of psychopathology. This study investigated expectable increases in statistical power by intense assessment in randomized controlled trials (RCTs). A simulation study, based on three scenarios and several empirical data sets, estimated power gains of two- or fivefold pre-post-assessment. For each condition, data sets of various effect sizes were generated, and AN(C)OVAs were applied to the sample of interest (N = 50-N = 200). Power increases ranged from 6% to 92%, with higher gains in more underpowered scenarios and with higher number of repeated assessments. ANCOVA profited from a more precise estimation of the baseline covariate, resulting in additional gains in statistical power. Fivefold pre-post EMA resulted in highest absolute statistical power and clearly outperformed traditional questionnaire assessments. For example, ANCOVA of automatized PHQ-9 questionnaire data resulted in absolute power of 55 (for N = 200 and d = 0.3). Fivefold EMA, however, resulted in power of 88.9. Non-parametric and multi-level analyses resulted in comparable outcomes. Besides providing psychological treatment, digital mental health can help optimizing sensitivity in RCT-based research. Intense assessment appears advisable whenever psychopathology needs to be assessed with high precision at pre- and post-assessment (e.g. small sample sizes, small treatment effects, or when applying optimization problems like machine learning). First empiric studies are promising, but more evidence is needed. Simulations for various effects and a short guide for popular power software are provided for study planning.

Interest & Impact

Downloads

330 since deposited on 19 May 2020
25 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item