Design, development and assessment of control schemes for IDMS in a standardized RTCP-based solution
Introduction
Research on media synchronization (sync, hereafter) has been primarily focused on orchestrating the playout of incoming media streams locally at single devices. On one hand, intra-stream sync deals with the maintenance of the temporal relationships between the Media Units (MUs), e.g., video frames or audio samples, of a specific media stream. On the other hand, inter-stream sync refers to the preservation of the temporal dependences between MUs of independent, but correlated, media streams, being lip-sync the most representative use case [1], [2]. Both types of media sync techniques are essential in distributed media applications.
Recently, novel forms of media sharing applications are gaining momentum [3], allowing geographically distributed users to socially interact (e.g., using text, audio or video chat) within the context of simultaneous content consumption. Some examples include Social TV, synchronous e-learning, and networked multi-player games. However, many challenges must be faced to enable truly interactive and coherent shared media experiences [4]. One of the major technological barriers is the lack of mechanisms to guarantee a simultaneous sync of the media playout across the involved devices, which is known as Inter-Destination Media Sync (IDMS) [3], [5], [6]. IDMS is needed to adaptively compensate the end-to-end (e2e) delay variability that is originated when delivering specific media content to a group of distributed devices. Indeed, previous studies have revealed that the magnitudes of these (playout) delay differences are significantly larger than acceptable limits in most of the IDMS use cases (e.g., [3], [7]), leading to a severe QoE (Quality of Experience) degradation [8], [9].
In [1], [10], the authors presented the design, implementation and evaluation of a preliminary version of a centralized IDMS solution, which is based on simple extensions to the RTP/RTCP standard protocols [11]. Experimental tests proved its satisfactory performance in both real, but controlled, scenarios [1] and simulated ones [10]. However, with the advent of distributed media consumption, further requirements emerge. First, inter-operability between (third-party) implementations and devices needs to be guaranteed. Second, some IDMS use cases require very strict sync levels [3], so highly accurate IDMS solutions must be provided. Furthermore, our study in [3] pointed out that a centralized approach is not always the best choice for IDMS, as in some cases distributed solutions are more suitable.
The main goal of this work is to meet the above requirements. Accordingly, the contribution of this paper is three-fold. First, novel standard compliant RTCP extensions1 for IDMS have been designed to guarantee compatibility and high accuracy. Second, our RTP/RTCP-based IDMS solution has been extended to be able to adopt different control or architectural schemes. This allows our IDMS solution to be adaptive and flexible enough to be efficiently deployed in different networked environments. Likewise, newer aspects and functionalities have been added to improve its performance, for each scheme in use. Third, our RTP/RTCP-based IDMS solution has been implemented in a simulation framework and its consistent behavior and satisfactory responsiveness are proved through simulation tests in a small scale setup. Further research will be targeted to analyze the scalability and adaptability of our IDMS solution by performing a thorough evaluation in large scale and dynamic settings.
The structure of the paper is as follows. In the next section, the state-of-the-art regarding IDMS is reviewed. In Section 3, we describe the main components of our IDMS solution, for each control scheme. Section 4 gives performance results. Finally, Section 5 provides our conclusions and a discussion of future work.
Section snippets
Related work
Due to the increasing relevancy of IDMS, several works have addressed this topic. In [5], authors presented a survey of IDMS solutions, while in [3] we presented a compilation of IDMS use cases. Besides, the recent work in [6] provides a historical review of multimedia sync techniques (including IDMS).
Two main categories of IDMS solutions can be distinguished (see [5], [6]): axis-based, which aim to continuously align the presentation of media streams along either a virtual or a wall-clock
Standardized RTP/RTCP-based IDMS solution
This section describes all the components of our RTP/RTCP-based IDMS solution and discusses the improvements that have been added. To aid understanding this paper, Table 2 lists the symbols defined in this section and their meaning.
Simulation scenario and setup
The development, modeling and simulations were conducted using NS-2 [33]. We have tested our IDMS solution in the small scale scenario shown in Fig. 7. The simulation setup is identical to the one in [10]. This is because we want to keep continuity, as well as to show the evolution, of results with respect our work in [10], in which the preliminary version of our SMS-based IDMS solution was evaluated. As a first objective, we aim to test the correct exchange of the newly designed RTCP messages,
Conclusions and future work
In this work, we have presented the design and development of an adaptive, accurate and standard compliant RTP/RTCP-based IDMS solution. Moreover, because of the suitability of different control schemes for specific use cases, our IDMS solution has been extended and adapted to adopt both centralized (SMS and M/S Scheme) and distributed schemes (DCS). This enhances the flexibility of our IDMS solution to be efficiently deployed in a wide range of scenarios. Simulation tests have shown the
Acknowledgments
This work has been financed, partially, by Universitat Politècnica de València (UPV), under its R&D Support Program in PAID-01-10. TNO’s work has been partially funded by European Community’s Seventh Framework Programme (FP7/2007–2013) under Grant Agreement No. ICT-2011-8-318343 (STEER Project). CWI’s work has been partially funded by the European Community’s Seventh Framework Programme (FP7/2007–2013) under Grant Agreement No. ICT-2011-7-287723 (REVERIE Project).
Mario Montagud was born in Montitxelvo (Spain). He studied Telecommunications Engineering at Polytechnic University of Valencia (UPV). Since then, he is researching about Computer Networks, Multimedia Systems, Synchronization Protocols and Simulation Techniques. He is (co-) author of over 20 scientific papers, is member of the Technical Committee of several international conferences and of the Editorial Board of international journals. Webpage: https://sites.google.com/site/mamontor/.
References (37)
- et al.
From IPTV to synchronous shared experiences challenges in design: distributed media synchronization
Signal Process.: Image Commun.
(2011) - et al.
Multimedia group and inter-stream synchronization techniques: a comparative study
Inform. Syst.
(2009) - et al.
Enhanced adaptive RTCP-based inter-destination multimedia synchronization approach for distributed applications
Comput. Netw. J.
(2012) - et al.
An RTP/RTCP based approach for multimedia group and inter-stream synchronization
Multimedia Tools Appl. J.
(2008) - et al.
Real-time adaptive content-based synchronization of multimedia streams
Adv. Multimedia J.
(2011) - et al.
Inter-destination multimedia synchronization: schemes, use cases and standardization
Multimedia Syst.
(2012) - et al.
Evolution of temporal multimedia synchronization principles: a historical viewpoint
ACM TOMCCAP
(2013) - O. van Deventer, H. Stokking, O. Niamut, F. Walraven, V. Klos, Advanced interactive television service require...
- D.A. Shamma, M. Bastea-Forte, N. Joubert, Y. Liu, Enhancing online personal connections through synchronized sharing of...
- D. Geerts, I. Vaishnavi, R. Mekuria, O. van Deventer, P. Cesar, Are we in sync?: synchronization requirements for...
Cited by (0)
Mario Montagud was born in Montitxelvo (Spain). He studied Telecommunications Engineering at Polytechnic University of Valencia (UPV). Since then, he is researching about Computer Networks, Multimedia Systems, Synchronization Protocols and Simulation Techniques. He is (co-) author of over 20 scientific papers, is member of the Technical Committee of several international conferences and of the Editorial Board of international journals. Webpage: https://sites.google.com/site/mamontor/.
Fernando Boronat was born in Gandia (Spain), and studied Telecommunications Engineering at the Polytechnic University of Valencia (UPV) in Spain. After working for several Spanish Telecommunication Companies, in 1996 he moved back to the UPV, where he is an Assistant Professor in the Communications Department at the Gandia Campus. He obtained his phD degree in 2004 and his main topics of interest are Communication Networks, Multimedia Systems, Multimedia Protocols and Synchronization. He is IEEE senior member (since 1993) and is involved in several IPCs of national and international journals and conferences. Webpage: http://personales.upv.es/fboronat/Research/index_investig_en.html.
Hans Stokking is a Senior Scientist and inventor at TNO in the field of Information and Communication Technology. His focus is on integrating various fields of study, and as such, Hans is active in the area of IPTV solutions, content distribution networks and home networks. His main focus is in ’how to get services across networks’ and he has consulted a wide range of customers on a range of services and networks: from consumer and business VoIP services to Internet services to P2P networks and inter-business messaging networks. Hans is (co-) author of over 20 popular and scientific papers, is (co-) inventor of more than 40 patent applications and has prepared many standardization contributions over the years. He is also a TPC member for the yearly ICIN conference, and is (co-) organizer of the international Media Synchronization Workshop series.
Pablo Cesar leads the Distributed and Interactive Systems group at CWI (The National Research Institute for Mathematics and Computer Science in the Netherlands). He has (co)-authored over 70 articles about multimedia systems and infrastructures, social media sharing, interactive media, multimedia content modeling, and user interaction. He has given tutorials about multimedia systems in prestigious conferences such as ACM Multimedia, CHI, and the WWW conference. Webpage: http://homepages.cwi.nl/~garcia.