Plan inference techniques have been used extensively to understand natural language dialogue. But as noted by Clark[5], language and communication are more than just utterances. This paper presents the problems that we have had to address
and the solutions that we have devised in designing a system to recognize intentions from information graphics. Our work is part of a larger project to develop an interactive natural language system that provides an alternative means
for individuals with sight-impairments to access the content of information graphics.
The work of the second author was supported by the National Science Foundation under Grant No. 0132821.