User-centered adaptive streaming of dynamic point clouds for virtual reality remote communication

Remote communication applications have become a necessity in a globalised and connectedworld exemplified by the popularity of video conferencing applications. In recent years, virtual reality (VR) remote communication applications have emerged that aim to deliver a greater sense of co-presence and immersion in a shared virtual space where users are able to navigate freely while employing both verbal and non-verbal communication. Such applications require a volumetric user representation and point clouds have emerged as a popular format to represent real-time user reconstructions. However, volumetric point clouds are challenging to deliver over bandwidth-limited networks owing to the large volume of data required for dynamic streams. Adaptive streaming is the process of segmenting an object spatially and temporally in order to optimize the delivery of content by prioritizing the quality of spatial segments that are visible from a given viewport. In this thesis, we embed our research in the application scenario of VR remote communication with real-time point cloud user reconstructions and explore adaptive delivery optimizations. Previous work in the field mainly focused on using the entire point cloud object as the unit of bandwidth allocation in scenes containing multiple point clouds. Other recent work has relied on computationally complex surface estimation in order to spatially segment the point cloud that is unsuited to real-time applications. This thesis focuses on investigating if a user-centred approach combined with a low complexity adaptive streaming method can improve the quality of experience of interacting with point cloud user reconstructions in VR remote communication. We focus on optimizing the delivery of a remote user’s reconstruction with spatial segmentation and surface orientation estimation in real-time combinedwith an auxiliary utility function to allocate the available bandwidth across available segments. The utility is defined based on the position and surface orientation of the point cloud segment and the position and orientation of the user’s viewport.

• View at Worldcat

Full Text ( Final Version , 26mb )