Testing small study effects in multivariate meta-analysis.

Hong, Chuan; Salanti, Georgia; Morton, Sally; Riley, Richard; Chu, Haitao; Kimmel, Stephen E; Chen, Yong (2020). Testing small study effects in multivariate meta-analysis. Biometrics, 76(4), pp. 1240-1250. The International Biometric Society 10.1111/biom.13342

Hong Biometrics 2020_AAM.pdf - Accepted Version
Available under License Publisher holds Copyright.

Download (603kB) | Preview
[img] Text
Hong_Biometrics_2020.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (715kB) | Request a copy

Small study effects occur when smaller studies show different, often larger, treatment effects than large ones, which may threaten the validity of systematic reviews and meta-analyses. The most well-known reasons for small study effects include publication bias, outcome reporting bias and clinical heterogeneity. Methods to account for small study effects in univariate meta-analysis have been extensively studied. However, detecting small study effects in a multivariate meta-analysis setting remains an untouched research area. One of the complications is that different types of selection processes can be involved in the reporting of multivariate outcomes. For example, some studies may be completely unpublished while others may selectively report multiple outcomes. In this paper, we propose a score test as an overall test of small study effects in multivariate meta-analysis. Two detailed case studies are given to demonstrate the advantage of the proposed test over various naive applications of univariate tests in practice. Through simulation studies, the proposed test is found to retain nominal Type I error rates with considerable power in moderate sample size settings. Finally, we also evaluate the concordance between the proposed test with the naive application of univariate tests by evaluating 44 systematic reviews with multiple outcomes from the Cochrane Database.

Item Type:

Journal Article (Original Article)


04 Faculty of Medicine > Pre-clinic Human Medicine > Institute of Social and Preventive Medicine (ISPM)

UniBE Contributor:

Salanti, Georgia


600 Technology > 610 Medicine & health
300 Social sciences, sociology & anthropology > 360 Social problems & social services




The International Biometric Society


[4] Swiss National Science Foundation




Andrea Flükiger-Flückiger

Date Deposited:

11 Aug 2020 18:45

Last Modified:

19 Feb 2021 18:24

Publisher DOI:


PubMed ID:


Uncontrolled Keywords:

comparative effectiveness research composite likelihood outcome reporting bias publication bias small study effect systematic review





Actions (login required)

Edit item Edit item
Provide Feedback