In the present study, we examined the distribution and processing of information over time in auditory and visual speech as
it is used in unimodal and bimodal word recognition. English consonant-vowel-consonant words representing all possible initial
consonants were presented as auditory, visual, or audiovisual speech in a gating task. The distribution of information over
time varied across and within features. Visual speech information was generally fully available early during the phoneme,
whereas auditory information was still accumulated. An audiovisual benefit was therefore already found early during the phoneme.
The nature of the audiovisual recognition benefit changed, however, as more of the phoneme was presented. More features benefited
at short gates rather than at longer ones. Visual speech information plays, therefore, a more important role early during
the phoneme rather than later. The results of the study showed the complex interplay of information across modalities and
time, since this is essential in determining the time course of audiovisual spoken-word recognition.
The present research was supported in part by doctoral grants from the University of California, Santa Cruz, to A.J. This
work was part of the doctoral dissertation of A.J. Portions of this work have been presented in the proceedings of the Auditory-Visual
Speech Processing International Conference, Parksville, Canada, July 2005. The authors thank James McQueen and three anonymous
reviewers for discussion of earlier drafts.