ReflectanceFusion: Diffusion-based text to SVBRDF Generation
CoRR(2024)
Abstract
We introduce Reflectance Diffusion, a new neural text-to-texture model
capable of generating high-fidelity SVBRDF maps from textual descriptions. Our
method leverages a tandem neural approach, consisting of two modules, to
accurately model the distribution of spatially varying reflectance as described
by text prompts. Initially, we employ a pre-trained stable diffusion 2 model to
generate a latent representation that informs the overall shape of the material
and serves as our backbone model. Then, our ReflectanceUNet enables fine-tuning
control over the material's physical appearance and generates SVBRDF maps.
ReflectanceUNet module is trained on an extensive dataset comprising
approximately 200,000 synthetic spatially varying materials. Our generative
SVBRDF diffusion model allows for the synthesis of multiple SVBRDF estimates
from a single textual input, offering users the possibility to choose the
output that best aligns with their requirements. We illustrate our method's
versatility by generating SVBRDF maps from a range of textual descriptions,
both specific and broad. Our ReflectanceUNet model can integrate optional
physical parameters, such as roughness and specularity, enhancing
customization. When the backbone module is fixed, the ReflectanceUNet module
refines the material, allowing direct edits to its physical attributes.
Comparative evaluations demonstrate that ReflectanceFusion achieves better
accuracy than existing text-to-material models, such as Text2Mat, while also
providing the benefits of editable and relightable SVBRDF maps.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined