Compared with the color histogram, where the position information of each pixel is ignored, a simplified color correlogram (SCC) representation encodes the spatial information explicitly and enables an estimation algorithm to recover the object orientation. This paper analyzes the capability of the SCC (in a kernel based framework) in detecting and estimating object motion and presents a principled way to obtain motion observable SCCs as object representations to achieve more reliable tracking. Extensive experimental results demonstrate the reliability of the tracking procedure using the proposed algorithm.