The TabletPC is an example of a new generation of user interface device where pen-based manipulation of information is integrated directly into a user’s workflow. Using the TabletPC's existing pen and electronic ink systems, a wide range of static documents can be created or annotated. While the facilities of the TabletPC are useful for creating virtual images containing ink that can be overlaid on text or picture context, there is little support for creating annotations of time-based content such as video. This article describes an annotation authoring model and interface for creating peer-level annotations to video media. Peer-level annotations allow existing content to be enriched with additional content annotations that can be co-presented with the original media. A system for creating a SMIL language document containing SVG-based annotations that exist along-side the visual content is described, along with a discussion of the needs and limitations of supporting video markup in a web context. An example using peer-level annotations in a medical context is provided.