VT-NeRF: Neural radiance field with a vertex-texture latent code for high-fidelity dynamic human-body rendering

The fusion of a human prior with neural rendering techniques has recently emerged as one of the most promising approaches to processing dynamic human-body scenes with sparse inputs. However, learning geometric details and appearance in dynamic human-body scenes based solely on a human prior model represents a severely under-constrained problem. A new human-body representation method to solve this problem: a neural radiance field with vertex-texture latent codes (VT-NeRF) is proposed. VT-NeRF uses joint latent code to improve access to detailed information, combining vertex latent codes with 2D texture latent codes for the body surface. Referencing a 3D human skeleton for accurate guidance, the human model can quickly match poses and learn information about the body in different frames. VT-NeRF can integrate body information from different frames and different poses quickly because it uses an information-rich human prior: a 3D human skeleton and parametric models. A 3D human scene is then presented as an implied field of density and colour. Experiments with the ZJU-MoCap dataset show that our method outperforms previous methods in terms of both novel-view synthesis and 3D human reconstruction quality. It is twice as fast as Neural Body, and its average accuracy reaches 95.9%.