Welcome!
To use the personalized features of this site, please log in or register.
If you have forgotten your username or password, we can help.
|
 |
A Unified Framework for Constructing Multimodal Experiments and Applications
| |
|
A Unified Framework for Constructing Multimodal Experiments and Applications
Adam Cheyer3 , Luc Julia3 and Jean-Claude Martin4 
| (3) |
SRI International, Menlo Park, Ca., USA |
| (4) |
LIMSI-CNRS, Orsay, France |
Abstract
Inspired by a Wizard of Oz (WOZ) simulation experiment, we developed a working prototype of a system that enables users to
interact with a map display through synergistic combinations of pen and voice. To address many of the issues raised by multimodal
fusion, our implementation employed a distributed multi-agent framework to coordinate parallel competition and cooperation
among processing components. Since then, the agent-based infrastructure has been enhanced with a collaboration technology,
creating a framework in which multiple humans and automated agents can naturally interact within the same graphical workspace.
Our current endeavour is the leveraging of this architecture to create a unified implementation framework for simultaneously
developing both WOZ simulated systems and their fully-automated counterparts. Bootstrapping effects made possible by such
an approach are illustrated by an experiment currently under way in our laboratory: as a naive subject draws, writes, and
speaks requests to a (simulated) interactive map, a hidden Wizard responds as efficiently as possible using our best fully-automated
system, through either standard graphical interface devices or multimodal combinations of pen and voice. The input choices
made by both subject and Wizard are invaluable, and the data collected from each can be applied directly to evaluating and
improving the automated part of the system.
Fulltext Preview (Small, Large)
 References secured to subscribers.
|
|
|
|
|
|