Why Evaluating Semantic Web Applications is Difficult

This position paper discusses our experience in evaluating our cultural search and annotation engine. We identify three aspects that determine the quality of a semantic web application as a whole, namely: the quality of data set, the quality of underlying search and inference software and the quality of the user interface. We argue that evaluation of semantic web applications is particularly difficult because of the strong interdependency between the three aspects.