Semantic incongruity influences response caution in audio‑visual integration

Steinweg, Benjamin; Mast, Fred (2016). Semantic incongruity influences response caution in audio‑visual integration. Experimental brain research, 235(1), pp. 349-363. Springer 10.1007/s00221-016-4796-0

[img] Text
art%3A10.1007%2Fs00221-016-4796-0.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (988kB) | Request a copy

Multisensory stimulus combinations trigger shorter reaction times (RTs) than individual single-modal-ity stimuli. It has been suggested that this inter-sensory facilitation effect is found exclusively for semantically congruent stimuli, because incongruity would prevent mul-tisensory integration. Here we provide evidence that the effect of incongruity is due to a change in response cau-tion rather than prevention of stimulus integration. In two experiments, participants performed two-alternative forced-choice decision tasks in which they categorized auditory stimuli, visual stimuli or audio-visual stimulus pairs. The pairs were either semantically congruent (e.g. ambulance image and horn sound) or incongruent (e.g. ambulance image and bell sound). Shorter RTs and violations of the race model inequality on congruent trials are in accordance with previous studies. However, Bayesian hierarchical drift diffusion analyses contradict former co-activation-based explanations of the effects of congruency. Instead, they show that longer RTs on incongruent compared to congru-ent trials are most likely the result of an incongruity cau-tion effect—more cautious response behaviour in face of semantically incongruent sensory input. Further, they show that response caution can be adjusted on a trial-by-trial basis depending on incoming information. Finally, stimu-lus modality influenced non-cognitive components of the response. We suggest that the combined stimulus energy from simultaneously presented stimuli reduces encoding time.

Item Type:

Journal Article (Original Article)

Division/Institute:

07 Faculty of Human Sciences > Institute of Psychology > Cognitive Psychology, Perception and Methodology

UniBE Contributor:

Mast, Fred

Subjects:

100 Philosophy > 150 Psychology

ISSN:

0014-4819

Publisher:

Springer

Funders:

[4] Swiss National Science Foundation

Language:

English

Submitter:

Fred Mast

Date Deposited:

25 Nov 2016 14:06

Last Modified:

12 Jan 2017 01:30

Publisher DOI:

10.1007/s00221-016-4796-0

Uncontrolled Keywords:

Audio-visual integration; Multisensory; Drift diffusion model; Semantic congruency; Reaction times

BORIS DOI:

10.7892/boris.89322

URI:

https://boris.unibe.ch/id/eprint/89322

Actions (login required)

Edit item Edit item
Provide Feedback