Models of object recognition in cortex have so far been mostly applied to tasks involving the recognition of isolated objects presented on blank backgrounds. However, ultimately models of the visual system have to prove themselves in real world object recognition tasks. Here we took a first step in this direction: We investigated the performance of the HMAX model of object recognition in cortex recently presented by Riesenhuber & Poggio [1],[2] on the task of face detection using natural images. We found that the standard version of hmax performs rather poorly on this task, due to the low specificity of the hardwired feature set of C2 units in the model (corresponding to neurons in intermediate visual area V4) that do not show any particular tuning for faces vs. background. We show how visual features of intermediate complexity can be learned in HMAX using a simple learning rule. Using this rule, hmax outperforms a classical machine vision face detection system presented in the literature. This suggests an important role for the set of features in intermediate visual areas in object recognition.