Toward Tiny and High-quality Facial Makeup with Data Amplify Learning
arxiv(2024)
摘要
Contemporary makeup approaches primarily hinge on unpaired learning
paradigms, yet they grapple with the challenges of inaccurate supervision
(e.g., face misalignment) and sophisticated facial prompts (including face
parsing, and landmark detection). These challenges prohibit low-cost deployment
of facial makeup models, especially on mobile devices. To solve above problems,
we propose a brand-new learning paradigm, termed "Data Amplify Learning (DAL),"
alongside a compact makeup model named "TinyBeauty." The core idea of DAL lies
in employing a Diffusion-based Data Amplifier (DDA) to "amplify" limited images
for the model training, thereby enabling accurate pixel-to-pixel supervision
with merely a handful of annotations. Two pivotal innovations in DDA facilitate
the above training approach: (1) A Residual Diffusion Model (RDM) is designed
to generate high-fidelity detail and circumvent the detail vanishing problem in
the vanilla diffusion models; (2) A Fine-Grained Makeup Module (FGMM) is
proposed to achieve precise makeup control and combination while retaining face
identity. Coupled with DAL, TinyBeauty necessitates merely 80K parameters to
achieve a state-of-the-art performance without intricate face prompts.
Meanwhile, TinyBeauty achieves a remarkable inference speed of up to 460 fps on
the iPhone 13. Extensive experiments show that DAL can produce highly
competitive makeup models using only 5 image pairs.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要