Media Synchronization in 3D tele-Immersion Applications: an Architecture

This article discusses the challenges ahead for assuring media synchronization in 3D tele-immersion applications. The discussion is based on an architecture and use cases that are defined in the European FP7 project REVERIE. The architecture allows capturing, transmission, and rendering, in real-time, various types of 3D media streams (e.g., geometry, movements of the participants, 3D audio), with the final objective of enabling immersive communication (and interactions) between remote users. For achieving the final goal, media synchronization is a key requirement. In particular, this article will focus on two types of synchronization: between real-time streams, and between downloaded and real-time streams. The first case refers to the synchronization needed between the different real-time captured media (e.g., 3D audio and visual streams). The second one aims at synchronizing downloaded content (e.g., 3D models) and the media captured in real-time. The solution reported in this article is implemented as a novel real-time streaming engine that can handle various types of 3D media streams. Moreover, based on global timestamps, the engine can