Evidence-synthesis methods for personalizing the choice of treatment.

Seo, Michael (2022). Evidence-synthesis methods for personalizing the choice of treatment. (Unpublished). (Dissertation, University of Bern, the Faculty of Medicine and the Faculty of Human Sciences)

Text 2022_Seo_Michael_PhD_Thesis.pdf - Other Restricted to registered users only Available under License BORIS Standard License. Download (1MB) | Request a copy

Background

This thesis comprises work done in several research areas, including meta-analysis, network
meta-analysis and prediction modelling. Below, I briefly provide some background for each
of these areas.
Meta-analysis of individual patient data (IPD) from randomized controlled trials (RCTs) can
potentially be used to identify whether treatment effects substantially differ across clinically
important subgroups and to potentially pinpoint the best treatment for each patient. Statistical
methods for IPD meta-analysis have been established. However, RCTs often collect
information on a large number of patient-level variables (covariates), some of which might be
unrelated to the outcome of interest. Including too many covariates in an IPD meta-analysis
model might lead to worse estimates, and might hinder interpretation of results. Currently
there is a lack of guidance on how to select covariates to include in an IPD meta-analysis
model.
In addition, there has been growing interest in using data from non-randomized studies (NRS)
to complement evidence from RCTs in medical decision-making. This is because, although
RCTs are the best source of evidence regarding relative treatment effects, they often employ
strict experimental settings, which may hamper their ability to predict outcomes in ‘realworld’
clinical settings. Currently, there is a gap in methods for combining IPD from RCTs
and NRS, when aiming to make patient-specific predictions about the real-world effects of
medical interventions.
Moreover, clinical prediction models are widely used in modern clinical practice. Such
models are often developed using IPD from a single study, but often there are IPD available
from multiple studies. This allows using meta-analytical methods for developing prediction
models, increasing power and precision. Different studies, however, often measure different
sets of predictors, which may result to systematically missing predictors, i.e., when not all
studies collect all predictors of interest. This situation poses challenges in model
development.
Finally, network meta-analysis (NMA) can be used to compare multiple competing
treatments for the same disease. In practice, usually a range of outcomes are of interest. As
the number of outcomes increases, summarizing results from multiple NMAs becomes a nontrivial
task, especially for larger networks. In addition, NMAs provide results in terms of
relative effect measures that can be difficult to interpret and apply in every-day clinical
practice, such as the odds ratios.

Aims

This thesis has four research aims.
The first aim was to explore whether a systematic approach to the selection of treatmentcovariate
interactions in an IPD meta-analysis can lead to better estimates of patient-specific
treatment effects.
The second aim was to describe a general framework for developing models that combine
individual patient data from RCTs and NRS when aiming to predict outcomes for a set of
competing medical interventions applied in real-world clinical settings.
The third aim was to explore approaches that can be used to develop prediction models for
continuous outcomes, when not all studies collect all predictors of interest, i.e. resulting in
systematically missing predictors.
The fourth aim was to facilitate the clinical decision-making process by proposing a new
graphical tool, the Kilim plot, for presenting results from NMA on multiple outcomes.
Methods
For the first aim, we compared in simulations the standard approach to IPD meta-analysis (no
variable selection, all treatment-covariate interactions included in the model) with six
alternative methods: stepwise regression, and five regression methods that perform shrinkage
on treatment-covariate interactions. To illustrate our methods, we used dataset from
cardiology comparing new generation drug-eluting and bare-metal stents for percutaneous
coronary intervention and from psychiatry comparing antidepressant treatment of major
depression.
For the second aim, we developed six meta-analytical models and a simpler model for
making predictions about patients in real world settings. We focused on Bayesian approaches
and utilized methods such as shrinkage, calibration of intercept and main effects of
covariates, and weighting approaches to account for different study designs. We used a
dataset of patients with rheumatoid arthritis obtained from three RCTs and two registries to
illustrate our methods.
For the third aim, we compared four approaches: a naïve approach, where the model is
developed using only predictors measured in all studies; a multiple imputation approach that
ignores patient allocation in studies; a multiple imputation approach that accounts for study
allocation; and a new approach that develops a prediction model in each study separately
using all predictors reported, and then synthesizes all predictions in a multi-study ensemble.
For the fourth aim, we worked on developing a new plot that compactly summarizes results
on all treatments and all outcomes; it provides information regarding the strength of the
statistical evidence of treatments, while it illustrates absolute, rather than relative, effects of
interventions.

Results

For the first aim, exploring a range of scenarios, we found in simulations that shrinkage
methods performed well for both continuous and dichotomous outcomes, for a variety of
settings. We exemplified all methods in two real examples and saw that using more advanced
methods may lead to different estimates of relative treatment effects.
For the second aim, we developed several evidence-synthesis models. We found that, for our
example, models that pool information from both RCTs and non-randomized studies might
provide the best predictions for patients in a new setting.
For the third aim, we found that in simulations existing multiple imputation methods and our
new method outperform the naïve approach. In several scenarios, our method outperformed
imputation methods, especially for few studies, when predictor effects were small, and in
case of large heterogeneity.
For the fourth aim, we developed the Kilim plot which provide a holistic view of the
available evidence expressed in terms of absolute treatment effects and their corresponding
strength of statistical evidence.

Conclusion

From the first project, we conclude that variable selection is essential in meta-analyzing IPD
from multiple RCTs, especially when there are many reported covariates. Both frequentist
and Bayesian variable selection methods can be used, as long as the information regarding
study allocation of patients in studies is included in the model.
In the second project, we saw that the gain in predictive performance obtained from models
combining RCTs and NRS was modest in our clinical example. Nevertheless, the illustration
of different modelling approaches and the considerations regarding different cross-validation
methods that we provide may be valuable to inform future studies aiming to predict realworld
outcomes of competing interventions.
Based on the results of the third project, we recommend researchers faced with systematically
missing predictors to select among the different methods after using both internal and
internal-external cross-validation approaches. We think that our new ensemble method offers
a potentially powerful alternative to researchers, and that it might be especially useful in the
common case of having IPD from only a handful of studies, reporting different sets of
predictors.
For the fourth aim, we conclude that the Kilim plot can be a valuable aid in summarizing and
communicating results from NMAs on multiple outcomes. It can be especially useful for
larger networks, for the case of many outcomes, and when aiming to communicate NMA
results with patients and/or clinicians, so as to facilitate every-day clinical practice.

Interest & Impact

Downloads

15 since deposited on 27 Jul 2022
15 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item