2021-03-01
Generative RGB-D face completion for head-mounted display removal
Publication
Publication
Head-mounted displays (HMDs) are an essential display device for the observation of virtual reality (VR) environments. However, HMDs obstruct external capturing methods from recording the user's upper face. This severely impacts social VR applications, such as teleconferencing, which commonly rely on external RGB-D sensors to capture a volumetric representation of the user. In this paper, we introduce an HMD removal framework based on generative adversarial networks (GANs), capable of jointly filling in missing color and depth data in RGB-D face images. Our framework includes an RGB-based identity loss function for identity preservation and several components aimed at surface reproduction. Our results demonstrate that our framework is able to remove HMDs from synthetic RGB-D face images while preserving the subject's identity.
| Additional Metadata | |
|---|---|
| , , , , , , , | |
| doi.org/10.1109/VRW52623.2021.00028 | |
| 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2021 | |
| Organisation | Distributed and Interactive Systems |
|
Numan, N., ter Haar, F., & César Garcia, P. S. (2021). Generative RGB-D face completion for head-mounted display removal. In Proceedings - 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops, VRW 2021 (pp. 109–116). doi:10.1109/VRW52623.2021.00028 |
|
