On the Effect of Inter-observer Variability for a Reliable Estimation of Uncertainty of Medical Image Segmentation

Jungo, Alain; Meier, Raphael; Ermis, Ekin; Blatti-Moreno, Marcela; Herrmann, Evelyn; Wiest, Roland; Reyes, Mauricio (2018). On the Effect of Inter-observer Variability for a Reliable Estimation of Uncertainty of Medical Image Segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Lecture Notes in Computer Science: Vol. 11070 (pp. 682-690). Springer 10.1007/978-3-030-00928-1_77

[img] Text
Jungo2018_Chapter_OnTheEffectOfInter-observerVar.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (1MB) | Request a copy

Uncertainty estimation methods are expected to improve the understanding and quality of computer-assisted methods used in medical applications (e.g., neurosurgical interventions, radiotherapy planning), where automated medical image segmentation is crucial. In supervised machine learning, a common practice to generate ground truth label data is to merge observer annotations. However, as many medical image tasks show a high inter-observer variability resulting from factors such as image quality, different levels of user expertise and domain knowledge, little is known as to how inter-observer variability and commonly used fusion methods affect the estimation of uncertainty of automated image segmentation. In this paper we analyze the effect of common image label fusion techniques on uncertainty estimation, and propose to learn the uncertainty among observers. The results highlight the negative effect of fusion methods applied in deep learning, to obtain reliable estimates of segmentation uncertainty. Additionally, we show that the learned observers’ uncertainty can be combined with current standard Monte Carlo dropout Bayesian neural networks to characterize uncertainty of model’s parameters.

Item Type:

Conference or Workshop Item (Paper)


04 Faculty of Medicine > Pre-clinic Human Medicine > Institute for Surgical Technology & Biomechanics ISTB [discontinued]
04 Faculty of Medicine > Department of Radiology, Neuroradiology and Nuclear Medicine (DRNN) > Institute of Diagnostic and Interventional Neuroradiology
04 Faculty of Medicine > Department of Haematology, Oncology, Infectious Diseases, Laboratory Medicine and Hospital Pharmacy (DOLS) > Clinic of Radiation Oncology

Graduate School:

Graduate School for Cellular and Biomedical Sciences (GCB)

UniBE Contributor:

Jungo, Alain, Meier, Raphael, Ermis, Ekin, Blatti, Marcela Judith, Herrmann, Evelyn, Wiest, Roland Gerhard Rudi, Reyes, Mauricio


500 Science > 570 Life sciences; biology
600 Technology > 610 Medicine & health




Lecture Notes in Computer Science






Alain Jungo

Date Deposited:

17 Sep 2019 13:05

Last Modified:

02 Mar 2023 23:32

Publisher DOI:


Additional Information:

Medical Image Computing and Computer Assisted Intervention – MICCAI 2018





Actions (login required)

Edit item Edit item
Provide Feedback