Acoi: A System for Indexing Multimedia Objects

In this paper, we present a system that combines independent feature detector programs with multimedia database technology to provide a semantic rich index to multimedia data items on the World Wide Web. First, we introduce a grammatical framework, called feature grammars, which forms the indexing schema. Feature grammars are an extension of context-free grammars with active symbols (e.g., multimedia feature detectors) that may invoke feature detector programs. The Acoi system reads in the grammar, compiles it and executes it against a data source, e.g. a multimedia object. The derived parse tree is used as an index to this data source. Its structure closely resembles that of semi-structured (XML) documents. Then, we present the architecture of our implementation on top of Monet, our extensible main memory database system. In this implementation, feature grammars are used as a description of the execution sequence of the indexing program. We show how the resulting parse tree can be stored efficiently in Monet. A SQL-like query language enables users to use the feature grammar as a schema for query formulation to retrieve both index values and the original data sources. Throughout the paper, we illustrate the concepts with a running example of a grammar used for indexing HTML pages and multimedia objects on the World Wide Web.