Implicit Style-Content Separation using B-LoRA
arxiv(2024)
摘要
Image stylization involves manipulating the visual appearance and texture
(style) of an image while preserving its underlying objects, structures, and
concepts (content). The separation of style and content is essential for
manipulating the image's style independently from its content, ensuring a
harmonious and visually pleasing result. Achieving this separation requires a
deep understanding of both the visual and semantic characteristics of images,
often necessitating the training of specialized models or employing heavy
optimization. In this paper, we introduce B-LoRA, a method that leverages LoRA
(Low-Rank Adaptation) to implicitly separate the style and content components
of a single image, facilitating various image stylization tasks. By analyzing
the architecture of SDXL combined with LoRA, we find that jointly learning the
LoRA weights of two specific blocks (referred to as B-LoRAs) achieves
style-content separation that cannot be achieved by training each B-LoRA
independently. Consolidating the training into only two blocks and separating
style and content allows for significantly improving style manipulation and
overcoming overfitting issues often associated with model fine-tuning. Once
trained, the two B-LoRAs can be used as independent components to allow various
image stylization tasks, including image style transfer, text-based image
stylization, consistent style generation, and style-content mixing.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要