In this paper, we describe a pilot experiment run at CLEF 2003 for cross language image retrieval, called ImageCLEF. The task is this: given a user need expressed in a language different from the document collection, find as many relevant images as possible. To facilitate retrieval, textual captions are associated with each image, thereby enabling (but not limiting) retrieval using text-based retrieval methods. This paper describes our experiences of building a test collection for the ImageCLEF task, discusses the results from this campaign, and outlines our ideas for further ImageCLEF experiments.