DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception.

Ghamsarian, Negin; Wolf, Sebastian; Zinkernagel, Martin; Schoeffmann, Klaus; Sznitman, Raphael (2024). DeepPyramid+: medical image segmentation using Pyramid View Fusion and Deformable Pyramid Reception. International journal of computer assisted radiology and surgery, 19(5), pp. 851-859. Springer 10.1007/s11548-023-03046-2

s11548-023-03046-2.pdf - Published Version
Available under License Creative Commons: Attribution (CC-BY).

Download (912kB) | Preview


Semantic segmentation plays a pivotal role in many applications related to medical image and video analysis. However, designing a neural network architecture for medical image and surgical video segmentation is challenging due to the diverse features of relevant classes, including heterogeneity, deformability, transparency, blunt boundaries, and various distortions. We propose a network architecture, DeepPyramid+, which addresses diverse challenges encountered in medical image and surgical video segmentation.


The proposed DeepPyramid+ incorporates two major modules, namely "Pyramid View Fusion" (PVF) and "Deformable Pyramid Reception" (DPR), to address the outlined challenges. PVF replicates a deduction process within the neural network, aligning with the human visual system, thereby enhancing the representation of relative information at each pixel position. Complementarily, DPR introduces shape- and scale-adaptive feature extraction techniques using dilated deformable convolutions, enhancing accuracy and robustness in handling heterogeneous classes and deformable shapes.


Extensive experiments conducted on diverse datasets, including endometriosis videos, MRI images, OCT scans, and cataract and laparoscopy videos, demonstrate the effectiveness of DeepPyramid+ in handling various challenges such as shape and scale variation, reflection, and blur degradation. DeepPyramid+ demonstrates significant improvements in segmentation performance, achieving up to a 3.65% increase in Dice coefficient for intra-domain segmentation and up to a 17% increase in Dice coefficient for cross-domain segmentation.


DeepPyramid+ consistently outperforms state-of-the-art networks across diverse modalities considering different backbone networks, showcasing its versatility. Accordingly, DeepPyramid+ emerges as a robust and effective solution, successfully overcoming the intricate challenges associated with relevant content segmentation in medical images and surgical videos. Its consistent performance and adaptability indicate its potential to enhance precision in computerized medical image and surgical video analysis applications.

Item Type:

Journal Article (Original Article)


10 Strategic Research Centers > ARTORG Center for Biomedical Engineering Research
10 Strategic Research Centers > ARTORG Center for Biomedical Engineering Research > ARTORG Center - AI in Medical Imaging Laboratory
04 Faculty of Medicine > Department of Head Organs and Neurology (DKNS) > Clinic of Ophthalmology

UniBE Contributor:

Ghamsarian, Negin, Wolf, Sebastian (B), Zinkernagel, Martin Sebastian, Sznitman, Raphael


500 Science > 570 Life sciences; biology
600 Technology > 610 Medicine & health








Pubmed Import

Date Deposited:

09 Jan 2024 10:15

Last Modified:

17 May 2024 00:13

Publisher DOI:


PubMed ID:


Uncontrolled Keywords:

Deformable Convolutions Deformable Pyramid Reception Dilated Convolutions Medical Images Neural Networks Pyramid View Fusion Semantic Segmentation Surgical Videos




Actions (login required)

Edit item Edit item
Provide Feedback