Image Translation as Diffusion Visual Programmers
CoRR(2024)
摘要
We introduce the novel Diffusion Visual Programmer (DVP), a neuro-symbolic
image translation framework. Our proposed DVP seamlessly embeds a
condition-flexible diffusion model within the GPT architecture, orchestrating a
coherent sequence of visual programs (i.e., computer vision models) for various
pro-symbolic steps, which span RoI identification, style transfer, and position
manipulation, facilitating transparent and controllable image translation
processes. Extensive experiments demonstrate DVP's remarkable performance,
surpassing concurrent arts. This success can be attributed to several key
features of DVP: First, DVP achieves condition-flexible translation via
instance normalization, enabling the model to eliminate sensitivity caused by
the manual guidance and optimally focus on textual descriptions for
high-quality content generation. Second, the framework enhances in-context
reasoning by deciphering intricate high-dimensional concepts in feature spaces
into more accessible low-dimensional symbols (e.g., [Prompt], [RoI object]),
allowing for localized, context-free editing while maintaining overall
coherence. Last but not least, DVP improves systemic controllability and
explainability by offering explicit symbolic representations at each
programming stage, empowering users to intuitively interpret and modify
results. Our research marks a substantial step towards harmonizing artificial
image translation processes with cognitive intelligence, promising broader
applications.
更多查看译文
关键词
Image translation,Diffusion model
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要