Recently, MPV(multiple perspective video) has become popular because of recent advancement in digital video technologies. Actually, video surveillance is currently a big application of MPV. In the case of the usage of a lot of video cameras, human’s keyword annotation is a tedious task, and automatic image recognition techniques takes an expensive cost to find what are taken in the MPV. Therefore, we need an effective way of capturing camare metadata and sharing annotation information among MPV data. In this paper, we propose a way of modeling and structuring MPV data based on several semantic relationships for browsing and retrieval.