|
Visual Influences on Speech Perception
Many studies have shown that the visual system can assert a strong influence on the perception of
speech. One striking example of this is the McGurk effect, which occurs when viewing the utterance
of one consonant while listening to a different consonant. The resulting auditory percept is affected
by the visual input, and subjects report hearing either a combination percept (for example
perceiving /bga/ while viewing a /ba/ utterance and hearing a /ga/) or a fusion percept (for
example perceiving /da/ when viewing /ga/ and listening to /ba/). This indicates that the visual
system is able to influence the perception of speech stimuli.
Our lab has performed or is in the process of performing several studies related to visual influences
on speech perception. This includes a parametric investigation of the McGurk effect across three
different vowel contexts and two different syllable types in order to better characterize changes in
the McGurk effect percepts through a variety of contexts. Currently, we are performing functional imaging
experiments in both normal and deaf subjects to better understand the neural bases of audio-visual interactions.
Future work will include further imaging experiments and formulation of a
computational model of the brain processes underlying visual influences on
speech perception.
|
