In this article we claim that the linguistic-centred view within hypermediasystems needs refinement through a semiotic-based approach before real interoperation between media can be achieved. We discuss the problems of visual signification for images and video in dynamic systems, in which users can access visual material in a non-linear fashion. We describe how semiotics can help overcome such problems, by allowing descriptions of the material on both denotative and connotative levels. Finally we propose an architecture for a dynamic semiotic-aware hypermedia system.

User Interfaces (acm H.5.2), Graphics Utilities (acm I.3.4), Applications (acm I.3.8)
Information (theme 2)
Information Systems [INS]
Human-centered Data Analysis

Nack, F.-M, & Hardman, L. (2002). Denotative and connotative semantics in hypermedia: proposal for a semiotic-aware architecture. Information Systems [INS]. CWI.