Assessing the Predictive Performance of Probabilistic Caries Risk Assessment Models: The Importance of Calibration.

Trottini, Mario; Campus, Guglielmo; Corridore, Denise; Cocco, Fabio; Cagetti, Maria Grazia; Vigo, M Isabel; Polimeni, Antonella; Bossù, Maurizio (2020). Assessing the Predictive Performance of Probabilistic Caries Risk Assessment Models: The Importance of Calibration. Caries research, 54(3), pp. 258-265. Karger 10.1159/000507276

Text 000507276.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (250kB) | Request a copy

Probabilistic caries risk assessment models (P-CRA), such as the Cariogram, are promising tools to planning treatments in order to control and prevent caries. The usefulness of these models for informing patients and medical decision-making depends on 2 properties known as discrimination and calibration. Current common assessment of P-CRA models, however, ignores calibration, and this can be misleading. The aim of this paper was to provide tools for a proper assessment of calibration of the P-CRA models and improve calibration when lacking. A combination of standard calibration tools (calibration plot, calibration in-the-large, and calibration slope) and 3 novel measures of calibration (the Calibration Index and 2 related metrics, E50 and E90) are proposed to evaluate if a P-CRA model is well calibrated. Moreover, an approach was proposed and validated using data from a previous follow-up study performed on children evaluated by means of a reduced Cariogram model; Platt scaling and isotonic regression were applied showing a lack of calibration. The use of the Cariogram overestimates the actual risk of new caries for forecast probabilities <0.5 and underestimates the risk for forecast probabilities >0.6. Both Platt scaling and isotonic regression were able to significantly improve the calibration of the reduced Cariogram model, preserving its discrimination properties. The average specificity and sensitivity for both Platt scaling and isotonic regression using the cut-off point p= 0.5 were >83 and their sum well exceeded 160. The benefits of the proposed calibration methods are promising, but further research in this field is required.

Interest & Impact

Downloads

0 since deposited on 13 Aug 2020
0 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item