As virtual reality systems achieve new heights of visual and auditory realism, the need for improving the underlying conceptual modelling facilities becomes increasingly apparent. The Ossa system provides a media-independent modelling environment based on a production system model that uses conceptual graphs to represent both the facts and the rules. Using conceptual graphs allows for interaction with the virtual world using multiple modalities (e.g. graphics and natural language). Conceptual graphs also allow for highly expressive facts and rules, and a diagrammatic programming technique. The motivation, design, and implementation of the Ossa system are discussed.