Previous communication provides important context for new communication in an interaction. In natural language interfaces, the discourse context represents and maintains information about what has been said before. When other modes of communication are also used, they must also contribute to the context. In this paper, we describe how information about the graphics can be represented and maintained in the graphics context. We are particularly interested in how the graphics context can be used to support finding referents for plural anaphora.