In a typical video surveillance framework, a single camera or a set of cameras monitor a scene in which human activities are carried out. In this paper, we propose a complementary framework where human activities can be analyzed under a subjective point of view. The idea is to represent the focus of attention of a person in the form of a 3D view frustum, and to insert it in a 3D representation of the scene. This leads to novel inferences and reasoning on the scene and the people acting in it. As a particular application of this proposed framework, we collect the information from the subjective view frusta in an Interest Map, i.e. a map that gathers in an effective and intuitive way which parts of the scene are observed more often in a defined time interval. The experimental results on standard benchmark data witness the goodness of the proposed framework, encouraging further efforts for the development of novel applications in the same direction.