TexSliders: Diffusion-Based Texture Editing in CLIP Space

Generative models have enabled intuitive image creation and manipulationusing natural language. In particular, diffusion models have recently shownremarkable results for natural image editing. In this work, we propose to applydiffusion techniques to edit textures, a specific class of images that are anessential part of 3D content creation pipelines. We analyze existing editingmethods and show that they are not directly applicable to textures, since theircommon underlying approach, manipulating attention maps, is unsuitable for thetexture domain. To address this, we propose a novel approach that insteadmanipulates CLIP image embeddings to condition the diffusion generation. Wedefine editing directions using simple text prompts (e.g., "aged wood" to "newwood") and map these to CLIP image embedding space using a texture prior, witha sampling-based approach that gives us identity-preserving directions in CLIPspace. To further improve identity preservation, we project these directions toa CLIP subspace that minimizes identity variations resulting from entangledtexture attributes. Our editing pipeline facilitates the creation of arbitrarysliders using natural language prompts only, with no ground-truth annotateddata necessary.