Publication Details

Availability: Click here to download the PDF (Acrobat Reader) file (15 pages).

Keywords: image annotation, image retrieval, multi-label learning.

Appeared in: ImageClef Lab of CLEF 2011 Conference on Multilingual and Multimodal Information Access Evaluation, Amsterdam, Netherlands, 2011.

Abstract: We participated both in the photo annotation and concept-based retrieval tasks of CLEF 2011. For the annotation task we developed visual, textual and multi-modal approaches using multi-label learning algorithms from the Mulan open source library. For the visual model we employed the ColorDescriptor software to extract visual features from the images using 7 descriptors and 2 detectors. For each combination of descriptor and detector a multi-label model is built using the Binary Relevance approach coupled with Random Forests as the base classifier. For the textual models we used the boolean bag-of-words representation, and applied stemming, stop words removal, and feature selection using the chi-squared-max method. The multi-label learning algorithm that yielded the best results in this case was Ensemble of Classifier Chains using Random Forests as base classifier. Our multi-modal approach was based on a hierarchical late-fusion scheme. For the concept based retrieval task we developed two different approaches. The first one is based on the concept relevance scores produced by the system we developed for the annotation task. It is a manual approach, because for each topic we manually selected the relevant topics and manually set the strength of their contribution to the final ranking produced by a general formula that combines topic relevance scores. The second approach is based solely on the sample images provided for each query and is therefore fully automated. In this approach only the textual information was used in a query-by-example framework.

Relevant Links: ImageClef 2011