A Question-Centric Model for Visual Question Answering in Medical Imaging

Vu, Minh H.; Löfstedt, Tommy; Nyholm, Tufve; Sznitman, Raphael (2020). A Question-Centric Model for Visual Question Answering in Medical Imaging. IEEE transactions on medical imaging, 39(9), pp. 2856-2868. Institute of Electrical and Electronics Engineers IEEE 10.1109/TMI.2020.2978284

[img] Text
09024133.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (2MB) | Request a copy

Deep learning methods have proven extremely effective at performing a variety of medical image analysis tasks. With their potential use in clinical routine, their lack of transparency has however been one of their few weak points, raising concerns regarding their behavior and failure modes. While most research to infer model behavior has focused on indirect strategies that estimate prediction uncertainties and visualize model support in the input image space, the ability to explicitly query a prediction model regarding its image content offers a more direct way to determine the behavior of trained models. To this end, we present a novel Visual Question Answering approach that allows an image to be queried by means of a written question. Experiments on a variety of medical and natural image datasets show that by fusing image and question features in a novel way, the proposed approach achieves an equal or higher accuracy compared to current methods.

Item Type:

Journal Article (Original Article)


10 Strategic Research Centers > ARTORG Center for Biomedical Engineering Research > ARTORG Center - AI in Medical Imaging Laboratory

UniBE Contributor:

Sznitman, Raphael


000 Computer science, knowledge & systems
600 Technology
600 Technology > 610 Medicine & health




Institute of Electrical and Electronics Engineers IEEE




Raphael Sznitman

Date Deposited:

13 May 2020 11:47

Last Modified:

05 Dec 2022 15:37

Publisher DOI:






Actions (login required)

Edit item Edit item
Provide Feedback