Decision Curve Analysis of In-Hospital Mortality Prediction Models: The Relative Value of Pre- and Intraoperative Data For Decision-Making.

Huber, Markus; Bello, Corina; Schober, Patrick; Filipovic, Mark G; Luedi, Markus M (2024). Decision Curve Analysis of In-Hospital Mortality Prediction Models: The Relative Value of Pre- and Intraoperative Data For Decision-Making. Anesthesia and analgesia, 139(3), pp. 617-628. Lippincott Williams & Wilkins 10.1213/ANE.0000000000006874

[img] Text
decision_curve_analysis_of_in_hospital_mortality.733.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (2MB)

BACKGROUND

Clinical prediction modeling plays a pivotal part in modern clinical care, particularly in predicting the risk of in-hospital mortality. Recent modeling efforts have focused on leveraging intraoperative data sources to improve model performance. However, the individual and collective benefit of pre- and intraoperative data for clinical decision-making remains unknown. We hypothesized that pre- and intraoperative predictors contribute equally to the net benefit in a decision curve analysis (DCA) of in-hospital mortality prediction models that include pre- and intraoperative predictors.

METHODS

Data from the VitalDB database featuring a subcohort of 6043 patients were used. A total of 141 predictors for in-hospital mortality were grouped into preoperative (demographics, intervention characteristics, and laboratory measurements) and intraoperative (laboratory and monitor data, drugs, and fluids) data. Prediction models using either preoperative, intraoperative, or all data were developed with multiple methods (logistic regression, neural network, random forest, gradient boosting machine, and a stacked learner). Predictive performance was evaluated by the area under the receiver-operating characteristic curve (AUROC) and under the precision-recall curve (AUPRC). Clinical utility was examined with a DCA in the predefined risk preference range (denoted by so-called treatment threshold probabilities) between 0% and 20%.

RESULTS

AUROC performance of the prediction models ranged from 0.53 to 0.78. AUPRC values ranged from 0.02 to 0.25 (compared to the incidence of 0.09 in our dataset) and high AUPRC values resulted from prediction models based on preoperative laboratory values. A DCA of pre- and intraoperative prediction models highlighted that preoperative data provide the largest overall benefit for decision-making, whereas intraoperative values provide only limited benefit for decision-making compared to preoperative data. While preoperative demographics, comorbidities, and surgery-related data provide the largest benefit for low treatment thresholds up to 5% to 10%, preoperative laboratory measurements become the dominant source for decision support for higher thresholds.

CONCLUSIONS

When it comes to predicting in-hospital mortality and subsequent decision-making, preoperative demographics, comorbidities, and surgery-related data provide the largest benefit for clinicians with risk-averse preferences, whereas preoperative laboratory values provide the largest benefit for decision-makers with more moderate risk preferences. Our decision-analytic investigation of different predictor categories moves beyond the question of whether certain predictors provide a benefit in traditional performance metrics (eg, AUROC). It offers a nuanced perspective on for whom these predictors might be beneficial in clinical decision-making. Follow-up studies requiring larger datasets and dedicated deep-learning models to handle continuous intraoperative data are essential to examine the robustness of our results.

Item Type:

Journal Article (Original Article)

Division/Institute:

04 Faculty of Medicine > Department of Intensive Care, Emergency Medicine and Anaesthesiology (DINA) > Clinic and Policlinic for Anaesthesiology and Pain Therapy
04 Faculty of Medicine > Department of Intensive Care, Emergency Medicine and Anaesthesiology (DINA) > Clinic and Policlinic for Anaesthesiology and Pain Therapy > Partial clinic Insel

UniBE Contributor:

Huber, Markus, Bello, Corina Manuela, Filipovic, Mark Georg, Lüdi, Markus

Subjects:

600 Technology > 610 Medicine & health

ISSN:

0003-2999

Publisher:

Lippincott Williams & Wilkins

Language:

English

Submitter:

Pubmed Import

Date Deposited:

06 Feb 2024 08:36

Last Modified:

18 Aug 2024 00:12

Publisher DOI:

10.1213/ANE.0000000000006874

PubMed ID:

38315623

BORIS DOI:

10.48350/192615

URI:

https://boris.unibe.ch/id/eprint/192615

Actions (login required)

Edit item Edit item
Provide Feedback