LoopGaussian: Creating 3D Cinemagraph with Multi-view Images via Eulerian Motion Field
CoRR(2024)
Abstract
Cinemagraph is a unique form of visual media that combines elements of still
photography and subtle motion to create a captivating experience. However, the
majority of videos generated by recent works lack depth information and are
confined to the constraints of 2D image space. In this paper, inspired by
significant progress in the field of novel view synthesis (NVS) achieved by 3D
Gaussian Splatting (3D-GS), we propose LoopGaussian to elevate cinemagraph from
2D image space to 3D space using 3D Gaussian modeling. To achieve this, we
first employ the 3D-GS method to reconstruct 3D Gaussian point clouds from
multi-view images of static scenes,incorporating shape regularization terms to
prevent blurring or artifacts caused by object deformation. We then adopt an
autoencoder tailored for 3D Gaussian to project it into feature space. To
maintain the local continuity of the scene, we devise SuperGaussian for
clustering based on the acquired features. By calculating the similarity
between clusters and employing a two-stage estimation method, we derive an
Eulerian motion field to describe velocities across the entire scene. The 3D
Gaussian points then move within the estimated Eulerian motion field. Through
bidirectional animation techniques, we ultimately generate a 3D Cinemagraph
that exhibits natural and seamlessly loopable dynamics. Experiment results
validate the effectiveness of our approach, demonstrating high-quality and
visually appealing scene generation.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined