A spatio-temporal memory is presented as indispensible representational prerequisite for the recognition of spatiotemporal gestalt. We assume that this structure is replicated on different processing-levels in the visual system mirroring its hierarchical structure. Thus, each level requires a different representation for spatio-temporal information.

As a first step, we present a two-layered architecture for the qualitative representation of motion trajectories: The vectorial layer is quite accurate and allows switches between deictic and intrinsic frame of reference. The propositional layer is more abstract and reveals similarities and regularities of motion paths which will be useful for motion prediction.

First psychophysical experiments indicate that information about direction and position are not stored independently but mereley in form of a spatio-temporal compound.