We present a perception-based paradigm for image retrieval. The central component of this paradigm is a query-concept learner, which can learn users’ subjective query concepts through an intelligent sampling process. We show that the learner can collect user feedback and use it to perform collaborative image annotation in addition to learning subjective query concepts. On the one hand, the improved annotation can help provide better initial keyword-search results to seed perception-based image retrieval. On the other hand, the more effective image-research results can further refine annotation quality. The users of the system collaboratively help improve search quality through the query-concept learner. Our empirical results show that an image retrieval system powered by this perception-based paradigm performs significantly better than traditional systems in search accuracy, in multimodal integration, and in capability for personalization.