3D Communication System Integrating 3D Reconstruction and Rendering Display

In order to solve the problem that the existing 2D video communication has a high bandwidth occupancy rate, the inability to present three-dimensional portraits of characters and the high dependence of three-dimensional projection technology on hardware, we propose a 3D communication framework that integrates 3D reconstruction and rendering display. The framework transmits the video stream to the video encoding module. The video coding module mainly completes the work of face detection, key point feature extraction and depth map feature coding. It is transmitted to the terminal device through the transmission protocol for video decoding and reconstruction. Then combined with the 3D rendering algorithm, virtual viewpoint synthesis and stereo image encoding are performed to realize the 3D reconstruction of the real face. The frame structure opens up the technical path from ordinary 2D video acquisition to 3D communication system implementation. Experiments show that the use of this communication framework for real face 3D reconstruction has higher accuracy and better information fusion. The reconstruction rate can reach 15fps.