The cellular phone system with internet connectivity has provide us with the mobility of telephony, computing and networking. We focus to the mobile “Intelligent Interfaces” with agents in this article as the successor of mobile computing and networking. This article presents the examples of mobile intelligent interfaces along with our experiences of realization and discuss for future challenges.

New paradigms of interface, such as the Perceptual User Interface (PUI) and the Real-World Oriented Interface (RWI), have been recently introduced following the flourishing years of the Graphical User Interface (GUI). PUI is a computer interface with perceptual sensors such as computer vision and speech recognition. These sensors work as the eyes and ears of the computer to interpret the user’s situation and intention from multi-modal interactions(figure 1). RWI is a computer system that exploits the interactions and contexts (figure 2) of users with yet-computerized objects in the real world such as a desk, a book, and stationary. It connects objects and events in the real world to the computer world, i.e. cyberspace.

Both PUI and RWI consume a lot of computing power for interface functions such as sensing and interpreting. This consumption can be greater than the power necessary to accomplish the main task. For instance, the PUI may needlessly view an empty office until someone enters. The RWI may uselessly watch the stationary objects in a room all day. Toy Interface is a novel modeled real-world oriented interface that uses objects with toy-like shapes and attributes as the interface between the real world and cyberspace (see Figure 4). Because toys successfully represent real world objects including humans and events, they can become very good tangible interfaces that provide an intuitive metaphor.