Semantic incongruity influences response caution in audio‑visual integration

Steinweg, Benjamin; Mast, Fred W. (2016). Semantic incongruity influences response caution in audio‑visual integration. Experimental brain research, 235(1), pp. 349-363. Springer 10.1007/s00221-016-4796-0

Preview

Text art%3A10.1007%2Fs00221-016-4796-0.pdf - Published Version Available under License Publisher holds Copyright. Download (988kB) | Preview

Multisensory stimulus combinations trigger shorter reaction times (RTs) than individual single-modal-ity stimuli. It has been suggested that this inter-sensory facilitation effect is found exclusively for semantically congruent stimuli, because incongruity would prevent mul-tisensory integration. Here we provide evidence that the effect of incongruity is due to a change in response cau-tion rather than prevention of stimulus integration. In two experiments, participants performed two-alternative forced-choice decision tasks in which they categorized auditory stimuli, visual stimuli or audio-visual stimulus pairs. The pairs were either semantically congruent (e.g. ambulance image and horn sound) or incongruent (e.g. ambulance image and bell sound). Shorter RTs and violations of the race model inequality on congruent trials are in accordance with previous studies. However, Bayesian hierarchical drift diffusion analyses contradict former co-activation-based explanations of the effects of congruency. Instead, they show that longer RTs on incongruent compared to congru-ent trials are most likely the result of an incongruity cau-tion effect—more cautious response behaviour in face of semantically incongruent sensory input. Further, they show that response caution can be adjusted on a trial-by-trial basis depending on incoming information. Finally, stimu-lus modality influenced non-cognitive components of the response. We suggest that the combined stimulus energy from simultaneously presented stimuli reduces encoding time.

Interest & Impact

Downloads

76 since deposited on 25 Nov 2016
35 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item