Multi-Task Deep Learning for the Detection of Lesions on 68Ga-PSMA PET/CT Imaging

Shi, Kuangyu; Xu, L.; Tetteh, G.; Gafita, A.; Haupt, Fabian; Afshar Oromieh, Ali; Eiber, M.; Menze, B.H.; Rominger, Axel Oliver (2018). Multi-Task Deep Learning for the Detection of Lesions on 68Ga-PSMA PET/CT Imaging. European journal of nuclear medicine and molecular imaging, 45(S1), S43-S43. Springer-Verlag

Purpose: The emerging PSMA targeted radionuclide therapy
provides an effective method for the treatment of advanced
metastatic prostate cancer. To optimize diagostics, therapy
monitoring and ultimately the theranostic benefit, it is urgently
needed to characterize all the lesions to target before the treatment.
However, this is extremely challenging considering the
factor that dozens of lesions of heterogenous size and uptake
may distribute in a variety of anatomical context with different
background. Until now, there is no successful computer-aided lesion detection methods for PSMA imaging. Methods: A cohort
of 71 patients with advanced metastatic prostate cancer
were scanned with 68Ga-PSMA-11 PET/CT. For proof-of-concept,
we focus on the detection and segmentation of bone &
lymph node lesions in the pelvic area. To train the network, the
bone and lymph node lesions were manually labelled by a nuclear
medicine expert. A multi-task deep learning architecture
(MulTi-Net) based on fully convolutional neural networks was
developed to detect the lesions. It aimed to first extract salient
features from PET and CT, the combined features would then
be adopted to automatically detect all the lesions in a 3D manner.
The framework contains five fully connected convolutional
layers and two sigmoid classification layers. In contrast to conventional
V-Net, it deleted pooling layer to avoid possible losing
of fine texture information. A cross-hair filter was integrated to
extensively reduce hyperparameters and speed up the training
procedure. An additional regularization was added to deal
with class imbalanced tumor-to-background ratio. For comparison,
the detection accuracy of conventional W-Net (cascaded
V-Nets) were calculated. Results: Compared with conventional
W-Net, the multi-task deep learning has improved the detection
precision from 72.8% to 90.2%, recall from 59.9% to 76.3% for
bone lesion. For the lymph node lesion (n=63), it improved the
detection precision from 57.8% to 81.4% ad recall from 44.1% to
62.6%. Conclusion: We proposed the first deep learning method
for automatic detection of lesions on 68Ga-PSMA-11 PET/CT
images. A multi-task deep learning method was developed to
improve the detection accuracy compared with conventional
W-Net. The preliminary test on pelvic area confirmed the potential
of deep learning methods. At the moment, more data is
being processed, since increasing the amount of training data
will further enhance the performance of the developed deep
learning methods.

Item Type:

Conference or Workshop Item (Abstract)


04 Faculty of Medicine > Department of Radiology, Neuroradiology and Nuclear Medicine (DRNN) > Clinic of Nuclear Medicine

UniBE Contributor:

Shi, Kuangyu; Haupt, Fabian; Afshar Oromieh, Ali and Rominger, Axel Oliver


600 Technology > 610 Medicine & health








Sabine Lanz

Date Deposited:

13 Jun 2019 09:56

Last Modified:

13 Jun 2019 09:56

Additional Information:



Actions (login required)

Edit item Edit item
Provide Feedback