Semantic incongruity influences response caution in audio‑visual integration

Steinweg, Benjamin; Mast, Fred (2016). Semantic incongruity influences response caution in audio‑visual integration. Experimental brain research, 235(1), pp. 349-363. Springer 10.1007/s00221-016-4796-0

[img] Text
art%3A10.1007%2Fs00221-016-4796-0.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (988kB) | Request a copy

Multisensory stimulus combinations trigger shorter reaction times (RTs) than individual single-modal-ity stimuli. It has been suggested that this inter-sensory facilitation effect is found exclusively for semantically congruent stimuli, because incongruity would prevent mul-tisensory integration. Here we provide evidence that the effect of incongruity is due to a change in response cau-tion rather than prevention of stimulus integration. In two experiments, participants performed two-alternative forced-choice decision tasks in which they categorized auditory stimuli, visual stimuli or audio-visual stimulus pairs. The pairs were either semantically congruent (e.g. ambulance image and horn sound) or incongruent (e.g. ambulance image and bell sound). Shorter RTs and violations of the race model inequality on congruent trials are in accordance with previous studies. However, Bayesian hierarchical drift diffusion analyses contradict former co-activation-based explanations of the effects of congruency. Instead, they show that longer RTs on incongruent compared to congru-ent trials are most likely the result of an incongruity cau-tion effect—more cautious response behaviour in face of semantically incongruent sensory input. Further, they show that response caution can be adjusted on a trial-by-trial basis depending on incoming information. Finally, stimu-lus modality influenced non-cognitive components of the response. We suggest that the combined stimulus energy from simultaneously presented stimuli reduces encoding time.

Item Type:

Journal Article (Original Article)


07 Faculty of Human Sciences > Institute of Psychology > Cognitive Psychology, Perception and Methodology

UniBE Contributor:

Mast, Fred


100 Philosophy > 150 Psychology






[4] Swiss National Science Foundation




Fred Mast

Date Deposited:

25 Nov 2016 14:06

Last Modified:

12 Jan 2017 01:30

Publisher DOI:


Uncontrolled Keywords:

Audio-visual integration; Multisensory; Drift diffusion model; Semantic congruency; Reaction times




Actions (login required)

Edit item Edit item
Provide Feedback