Known-item retrieval on broadcast TV

Many content-based, multimedia retrieval systems are based on a feature-oriented approach to querying, mostly exposing a fixed set of features (introduced at design time) for querying purposes. This restriction to a limited set of features is problematic for two reasons: it restricts the expressiveness at the semantic level, and it seems unfeasible to obtain (a-priori) a sufficiently powerful set of features for all possible queries. We describe an alternative approach where users specify precisely the distinguishable characteristics of the desired result set. In this query process, the user first describes a representation of the content (based on a feature or collection of features) and then tells the system how to apply the representation in the search. Our prototype video retrieval system allows the expression of such queries as a sequence of operations, on MPEG video and audio streams, that can be executed on our database system. While the low-level decompression stage is implemented in an imperative programming language, the actual retrieval approach is expressed in declarative database queries. We assessed this system with a case study in known-item retrieval on broadcast video streams: detecting news bulletins in the stream, with the help of both audio and video information.