Volume 72, Number 1, 209-225, DOI: 10.3758/APP.72.1.209

Published in partnership with

Logo

The Psychonomic Society

The temporal distribution of information in audiovisual spoken-word identification

Alexandra Jesse and Dominic W. Massaro

View Related Documents

Abstract

In the present study, we examined the distribution and processing of information over time in auditory and visual speech as it is used in unimodal and bimodal word recognition. English consonant-vowel-consonant words representing all possible initial consonants were presented as auditory, visual, or audiovisual speech in a gating task. The distribution of information over time varied across and within features. Visual speech information was generally fully available early during the phoneme, whereas auditory information was still accumulated. An audiovisual benefit was therefore already found early during the phoneme. The nature of the audiovisual recognition benefit changed, however, as more of the phoneme was presented. More features benefited at short gates rather than at longer ones. Visual speech information plays, therefore, a more important role early during the phoneme rather than later. The results of the study showed the complex interplay of information across modalities and time, since this is essential in determining the time course of audiovisual spoken-word recognition.
The present research was supported in part by doctoral grants from the University of California, Santa Cruz, to A.J. This work was part of the doctoral dissertation of A.J. Portions of this work have been presented in the proceedings of the Auditory-Visual Speech Processing International Conference, Parksville, Canada, July 2005. The authors thank James McQueen and three anonymous reviewers for discussion of earlier drafts.

Fulltext Preview

Image of the first page of the fulltext document