In this paper, we consider the problem of detecting the faces without constrained input conditions such as backgrounds, luminance and different image quality. We have developed an efficient and automatic faces detection algorithm in color images. Both the skin-tone model and elliptical shape of faces are used to reduce the influence of environments. A pre-built skin color model is based on 2D Gaussian distribution and sample faces for the skin-tone model. Our face detection algorithm consists of three stages: skin-tone segmentation, candidate region extraction and face region decision. First, we scan entire input images to extract facial color-range pixels by pre-built skin-tone model from YCbCr color space. Second, we extract candidate face regions by using elliptical feature characteristic of the face. We apply the best-fit ellipse algorithm for each skin-tone region and extract candidate regions by applying required ellipse parameters. Finally, we use the neural network on each candidate region in order to decide real face regions. The proposed algorithm utilizes the momentum back-propagation model to train it for 20*20 pixel patterns.