This paper presents a new Bayesian face tracking method under particle filter framework. First, two adaptive feature models are proposed to extract face features from image sequences. Then the robustness of face tracking is reinforced via building a local dual closed loop model (LDCLM). Meanwhile, trajectory analysis, which helps to avoid unnecessary restarting of detection module, is introduced to keep tracked faces’ identity as consistent as possible. Experimental results demonstrate the efficacy of our method.