Experimental result analysis for a generative probabilistic image retrieval model

The main conclusion from the metrics-based evaluation of video retrieval systems at TREC's video track is that non-interactive image retrieval from general collections using visual information only is not yet feasible. We show how a detailed analysis of retrieval results -- looking beyond mean average precision (MAP) scores on topical relevance -- gives significant insight in the main problems with the visual part of the retrieval model under study. Such an analytical approach proves an important addition to standard evaluation measures. We investigate (informally) two aspects of the results of a generative probabilistic image retrieval model on the video track search task: how is image similarity captured and how do the visual results contribute to the MAP score. We then take a closer look at the ability of the retrieval model to capture both colour and texture information, and investigate the influence of model building initialisation on the retrieval results. We demonstrate that colour is predominant over texture in the current model, once more showing the difficulty in combining evidence from different sources of information. A final experiment demonstrates that, although the model building process is sensitive to its (random) initialisation, this does not harm retrieval results.