Tascon-Morales, Sergio; Márquez-Neila, Pablo; Sznitman, Raphael (2022). Consistency-preserving Visual Question Answering in Medical Imaging. Lecture notes in computer science, 13438, pp. 386-395. Springer 10.1007/978-3-031-16452-1_37
|
Text
paper.pdf - Accepted Version Available under License Publisher holds Copyright. Download (7MB) | Preview |
|
Text
978-3-031-16452-1_37.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (1MB) |
Visual Question Answering (VQA) models take an image and a natural-language question as input and infer the answer to the question. Recently, VQA systems in medical imaging have gained popularity thanks to potential advantages such as patient engagement and second opinions for clinicians. While most research efforts have been focused on improving architectures and overcoming data-related limitations, answer consistency has been overlooked even though it plays a critical role in establishing trustworthy models. In this work, we propose a novel loss function and corresponding training procedure that allows the inclusion of relations between questions into the training process. Specifically, we consider the case where implications between perception and reasoning questions are known a-priori. To show the benefits of our approach, we evaluate it on the clinically relevant task of Diabetic Macular Edema (DME) staging from fundus imaging. Our experiments show that our method outperforms state-of-the-art baselines, not only by improving model consistency, but also in terms of overall model accuracy. Our code and data are available at https://github.com/sergiotasconmorales/consistency_vqa.
Item Type: |
Conference or Workshop Item (Paper) |
---|---|
Division/Institute: |
10 Strategic Research Centers > ARTORG Center for Biomedical Engineering Research > ARTORG Center - AI in Medical Imaging Laboratory |
Graduate School: |
Graduate School for Cellular and Biomedical Sciences (GCB) |
UniBE Contributor: |
Tascon Morales, Sergio, Márquez Neila, Pablo, Sznitman, Raphael |
Subjects: |
500 Science > 570 Life sciences; biology 600 Technology > 610 Medicine & health |
ISSN: |
0302-9743 |
ISBN: |
978-3-031-16451-4 |
Series: |
LNCS |
Publisher: |
Springer |
Funders: |
[4] Swiss National Science Foundation |
Language: |
English |
Submitter: |
Sergio Tascon Morales |
Date Deposited: |
08 Jul 2022 08:07 |
Last Modified: |
31 Jul 2023 07:50 |
Publisher DOI: |
10.1007/978-3-031-16452-1_37 |
Related URLs: |
|
ArXiv ID: |
2206.13296 |
BORIS DOI: |
10.48350/171140 |
URI: |
https://boris.unibe.ch/id/eprint/171140 |