Predicting outcomes at the individual patient level: what is the best method?

Liu, Qiang; Ostinelli, Edoardo Giuseppe; De Crescenzo, Franco; Li, Zhenpeng; Tomlinson, Anneka; Salanti, Georgia; Cipriani, Andrea; Efthimiou, Orestis (2023). Predicting outcomes at the individual patient level: what is the best method? BMJ mental health, 26(1), pp. 1-6. BMJ 10.1136/bmjment-2023-300701

Preview

Text e300701.full.pdf - Published Version Available under License Creative Commons: Attribution (CC-BY). Download (745kB) | Preview

OBJECTIVE

When developing prediction models, researchers commonly employ a single model which uses all the available data (end-to-end approach). Alternatively, a similarity-based approach has been previously proposed, in which patients with similar clinical characteristics are first grouped into clusters, then prediction models are developed within each cluster. The potential advantage of the similarity-based approach is that it may better address heterogeneity in patient characteristics. However, it remains unclear whether it improves the overall predictive performance. We illustrate the similarity-based approach using data from people with depression and empirically compare its performance with the end-to-end approach.

METHODS

We used primary care data collected in general practices in the UK. Using 31 predefined baseline variables, we aimed to predict the severity of depressive symptoms, measured by Patient Health Questionnaire-9, 60 days after initiation of antidepressant treatment. Following the similarity-based approach, we used k-means to cluster patients based on their baseline characteristics. We derived the optimal number of clusters using the Silhouette coefficient. We used ridge regression to build prediction models in both approaches. To compare the models' performance, we calculated the mean absolute error (MAE) and the coefficient of determination (R2) using bootstrapping.

RESULTS

We analysed data from 16 384 patients. The end-to-end approach resulted in an MAE of 4.64 and R2 of 0.20. The best-performing similarity-based model was for four clusters, with MAE of 4.65 and R2 of 0.19.

CONCLUSIONS

The end-to-end and the similarity-based model yielded comparable performance. Due to its simplicity, the end-to-end approach can be favoured when using demographic and clinical data to build prediction models on pharmacological treatments for depression.

Interest & Impact

Downloads

4 since deposited on 15 Jun 2023
4 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item