A probabilistic system that retrieves face images based on verbal descriptions given by users is proposed. This interactive system prompts the user at each stage of query to provide a description about a facial feature that will help it to retrieve the required face image. It is a soft biometric system that groups people by the description of their faces. The method proposed automates the process of extracting general verbal descriptions of faces like ’long nosed’ or ’blonde haired’ and performs queries on them. The proposed method uses Bayesian learning for the query process and hence is more immune to errors in both the extraction of semantic descriptions and user given information. It was found that the required face image appeared 77.6% of the time within the top 5 and 90.4% of the time within the top 10 retrieved face images.