Welcome!
To use the personalized features of this site, please log in or register.
If you have forgotten your username or password, we can help.
My Menu
Saved Items

A Unified Framework for Constructing Multimodal Experiments and Applications

Adam CheyerContact Information, Luc JuliaContact Information and Jean-Claude MartinContact Information

(3)  SRI International, Menlo Park, Ca., USA
(4)  LIMSI-CNRS, Orsay, France
Abstract
Inspired by a Wizard of Oz (WOZ) simulation experiment, we developed a working prototype of a system that enables users to interact with a map display through synergistic combinations of pen and voice. To address many of the issues raised by multimodal fusion, our implementation employed a distributed multi-agent framework to coordinate parallel competition and cooperation among processing components. Since then, the agent-based infrastructure has been enhanced with a collaboration technology, creating a framework in which multiple humans and automated agents can naturally interact within the same graphical workspace.
Our current endeavour is the leveraging of this architecture to create a unified implementation framework for simultaneously developing both WOZ simulated systems and their fully-automated counterparts. Bootstrapping effects made possible by such an approach are illustrated by an experiment currently under way in our laboratory: as a naive subject draws, writes, and speaks requests to a (simulated) interactive map, a hidden Wizard responds as efficiently as possible using our best fully-automated system, through either standard graphical interface devices or multimodal combinations of pen and voice. The input choices made by both subject and Wizard are invaluable, and the data collected from each can be applied directly to evaluating and improving the automated part of the system.

Contact Information Adam Cheyer
Email: cheyer@speech.sri.com

Contact Information Luc Julia
Email: julia@speech.sri.com

Contact Information Jean-Claude Martin
Email: martin@limsi.fr
Fulltext Preview (Small, Large)
Image of the first page of the fulltext

References secured to subscribers.



Export this chapter
Export this chapter as RIS | Text
 
Referenced by
1 newer article

  1. Xu, Yong (2009) WOZ experiments for understanding mutual adaptation. AI & Society 23(2)
    [CrossRef]
Remote Address: 38.107.191.109 • Server: mpweb15
HTTP User Agent: CCBot/1.0 (+http://www.commoncrawl.org/bot.html)