Interpretability-Guided Content-Based Medical Image Retrieval

Silva, Wilson; Pöllinger, Alexander; Cardoso, Jaime S.; Reyes, Mauricio (2020). Interpretability-Guided Content-Based Medical Image Retrieval. Lecture notes in computer science, 12261, pp. 305-314. Cham: Springer 10.1007/978-3-030-59710-8_30

Full text not available from this repository. (Request a copy)

When encountering a dubious diagnostic case, radiologists typically search in public or internal databases for similar cases that would help them in their decision-making process. This search represents a massive burden to their workflow, as it considerably reduces their time to diagnose new cases. It is, therefore, of utter importance to replace this manual intensive search with an automatic content-based image retrieval system. However, general content-based image retrieval systems are often not helpful in the context of medical imaging since they do not consider the fact that relevant information in medical images is typically spatially constricted. In this work, we explore the use of interpretability methods to localize relevant regions of images, leading to more focused feature representations, and, therefore, to improved medical image retrieval. As a proof-of-concept, experiments were conducted using a publicly available Chest X-ray dataset, with results showing that the proposed interpretability-guided image retrieval translates better the similarity measure of an experienced radiologist than state-of-the-art image retrieval methods. Furthermore, it also improves the class-consistency of top retrieved results, and enhances the interpretability of the whole system, by accompanying the retrieval with visual explanations.

Item Type:

Conference or Workshop Item (Paper)


10 Strategic Research Centers > ARTORG Center for Biomedical Engineering Research > ARTORG Center - AI in Medical Imaging Laboratory
04 Faculty of Medicine > Department of Radiology, Neuroradiology and Nuclear Medicine (DRNN) > Institute of Diagnostic, Interventional and Paediatric Radiology
10 Strategic Research Centers > ARTORG Center for Biomedical Engineering Research

UniBE Contributor:

Pöllinger, Alexander and Reyes, Mauricio


000 Computer science, knowledge & systems
500 Science > 570 Life sciences; biology
600 Technology > 610 Medicine & health
600 Technology > 620 Engineering










Mauricio Antonio Reyes Aguirre

Date Deposited:

01 Feb 2021 14:40

Last Modified:

01 Feb 2021 14:40

Publisher DOI:



Actions (login required)

Edit item Edit item
Provide Feedback