Stochastic Segmentation with Conditional Categorical Diffusion Models

Zbinden, Lukas; Doorenbos, Lars; Pissas, Theodoros; Huber, Adrian Thomas; Sznitman, Raphael; Márquez Neila, Pablo (2023). Stochastic Segmentation with Conditional Categorical Diffusion Models. In: International Conference on Computer Vision (ICCV) 2023. arxiv. Cornell University 10.48550/arXiv.2303.08888

[img] Text
2303.08888_zbinden_iccv2023_paper.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (4MB) | Request a copy

Semantic segmentation has made significant progress in recent years thanks to deep neural networks, but the common objective of generating a single segmentation output that accurately matches the image's content may not be suitable for safety-critical domains such as medical diagnostics and autonomous driving. Instead, multiple possible correct segmentation maps may be required to reflect the true distribution of annotation maps. In this context, stochastic semantic segmentation methods must learn to predict conditional distributions of labels given the image, but this is challenging due to the typically multimodal distributions, high-dimensional output spaces, and limited annotation data. To address these challenges, we propose a conditional categorical diffusion model (CCDM) for semantic segmentation based on Denoising Diffusion Probabilistic Models. Our model is conditioned to the input image, enabling it to generate multiple segmentation label maps that account for the aleatoric uncertainty arising from divergent ground truth annotations. Our experimental results show that CCDM achieves state-of-the-art performance on LIDC, a stochastic semantic segmentation dataset, and outperforms established baselines on the classical segmentation dataset Cityscapes.

Item Type:

Conference or Workshop Item (Paper)


04 Faculty of Medicine > Department of Radiology, Neuroradiology and Nuclear Medicine (DRNN) > Institute of Diagnostic, Interventional and Paediatric Radiology
10 Strategic Research Centers > ARTORG Center for Biomedical Engineering Research > ARTORG Center - AI in Medical Imaging Laboratory
10 Strategic Research Centers > ARTORG Center for Biomedical Engineering Research

Graduate School:

Graduate School for Cellular and Biomedical Sciences (GCB)

UniBE Contributor:

Zbinden, Lukas, Doorenbos, Lars Jelte, Pissas, Theodoros, Huber, Adrian Thomas, Sznitman, Raphael, Márquez Neila, Pablo


600 Technology > 610 Medicine & health
000 Computer science, knowledge & systems
600 Technology




Cornell University




Lukas Zbinden

Date Deposited:

14 Jul 2023 14:17

Last Modified:

11 Oct 2023 12:28

Publisher DOI:


ArXiv ID:





Actions (login required)

Edit item Edit item
Provide Feedback