This paper presents an algorithm for the detection and tracking of moving objects based on color and texture analysis for real time processing. Our goal is to study human interaction by tracking people and objects. The object detection algorithm is based on color histograms and iteratively divided interest regions for motion detection. The tracking algorithm is based on correlograms which combines spectral and spatial information to match detected objects in consecutive frames.