A forward search algorithm for detecting extreme study effects in network meta-analysis.

Petropoulou, Maria; Salanti, Georgia; Rücker, Gerta; Schwarzer, Guido; Moustaki, Irini; Mavridis, Dimitris (2021). A forward search algorithm for detecting extreme study effects in network meta-analysis. Statistics in medicine, 40(25), pp. 5642-5656. Wiley-Blackwell 10.1002/sim.9145

Petropoulou_StatMed_2021_epub.pdf - Published Version
Available under License Creative Commons: Attribution (CC-BY).

Download (1MB) | Preview

In a quantitative synthesis of studies via meta-analysis, it is possible that some studies provide a markedly different relative treatment effect or have a large impact on the summary estimate and/or heterogeneity. Extreme study effects (outliers) can be detected visually with forest/funnel plots and by using statistical outlying detection methods. A forward search (FS) algorithm is a common outlying diagnostic tool recently extended to meta-analysis. FS starts by fitting the assumed model to a subset of the data which is gradually incremented by adding the remaining studies according to their closeness to the postulated data-generating model. At each step of the algorithm, parameter estimates, measures of fit (residuals, likelihood contributions), and test statistics are being monitored and their sharp changes are used as an indication for outliers. In this article, we extend the FS algorithm to network meta-analysis (NMA). In NMA, visualization of outliers is more challenging due to the multivariate nature of the data and the fact that studies contribute both directly and indirectly to the network estimates. Outliers are expected to contribute not only to heterogeneity but also to inconsistency, compromising the NMA results. The FS algorithm was applied to real and artificial networks of interventions that include outliers. We developed an R package (NMAoutlier) to allow replication and dissemination of the proposed method. We conclude that the FS algorithm is a visual diagnostic tool that helps to identify studies that are a potential source of heterogeneity and inconsistency.

Item Type:

Journal Article (Original Article)


04 Faculty of Medicine > Pre-clinic Human Medicine > Institute of Social and Preventive Medicine (ISPM)

UniBE Contributor:

Salanti, Georgia


600 Technology > 610 Medicine & health
300 Social sciences, sociology & anthropology > 360 Social problems & social services








Andrea Flükiger-Flückiger

Date Deposited:

28 Jul 2021 10:46

Last Modified:

25 Mar 2022 02:32

Publisher DOI:


PubMed ID:


Uncontrolled Keywords:

Cook's distance NMAoutlier forward search network meta-analysis outliers





Actions (login required)

Edit item Edit item
Provide Feedback