Evaluation axes for medical image retrieval systems - The ImageCLEF experience

Henning Müller, Paul Clough, William Hersh, Thomas Deselaers, Thomas Lehmann, Antoine Geissbuhler

Research output: Chapter in Book/Report/Conference proceedingConference contribution

21 Scopus citations


Content-based image retrieval in the medical domain is an extremely hot topic in medical imaging as it promises to help better managing the large amount of medical images being produced. Applications are mainly expected in the field of medical teaching files and for research projects, where performance issues and speed are less critical than in the field of diagnostic aid. Final goal with most impact will be the use as a diagnostic aid in a real-world clinical setting. Other applications of image retrieval and image classification can be the automatic annotation of images with basic concepts or the control of DICOM header information. ImageCLEF is part of the Cross Language Evaluation Forum (CLEF). Since 2004, a medical image retrieval task has been added. Goal is to create databases of a realistic and useful size and also query topics that are based on real-world needs in the medical domain but still correspond to the limited capabilities of purely visual retrieval at the moment. Goal is to direct the research onto real applications and towards real clinical problems to give researchers who are not directly linked to medical facilities a possibility to work on the interesting problem of medical image retrieval based on real data sets and problems. The missing link between computer science research departments and clinical routine is one of the biggest problems that becomes evident when reading much of the current literature on medical image retrieval. Most databases are extremely small, the treated problems often far from clinical reality, and there is no integration of the prototypes into a hospital infrastructure. Only few retrieval articles specifically mention problems related to the DICOM format (Digital Imaging and Communications in Medicine) and the sheer amount of data that needs to be treated in an image archive (> 30.000 images per day in the Geneva radiology). This article develops the various axes that can be taken into account for medical image retrieval system evaluation. First, the axes are developed based on current challenges and experiences from ImageCLEF. Then, the resources developed for ImageCLEF are listed and finally, the application of the axes is explained to show the bases of the Im-ageCLEFmed evaluation campaign. This article will only concentrate on the medical retrieval tasks, the non-medical tasks will only shortly be mentioned.

Original languageEnglish (US)
Title of host publicationProceedings of the 13th ACM International Conference on Multimedia, MM 2005
Number of pages9
StatePublished - 2005
Event13th ACM International Conference on Multimedia, MM 2005 - Singapore, Singapore
Duration: Nov 6 2005Nov 11 2005

Publication series

NameProceedings of the 13th ACM International Conference on Multimedia, MM 2005


Other13th ACM International Conference on Multimedia, MM 2005


  • Benchmarking
  • Content-based image retrieval
  • Evaluation
  • Image retrieval
  • Medical image retrieval

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Human-Computer Interaction
  • Software


Dive into the research topics of 'Evaluation axes for medical image retrieval systems - The ImageCLEF experience'. Together they form a unique fingerprint.

Cite this