Towards Codable Watermarking for Injecting Multi-bits Information to LLMs
arxiv(2023)
摘要
As large language models (LLMs) generate texts with increasing fluency and
realism, there is a growing need to identify the source of texts to prevent the
abuse of LLMs. Text watermarking techniques have proven reliable in
distinguishing whether a text is generated by LLMs by injecting hidden
patterns. However, we argue that existing LLM watermarking methods are
encoding-inefficient and cannot flexibly meet the diverse information encoding
needs (such as encoding model version, generation time, user id, etc.). In this
work, we conduct the first systematic study on the topic of Codable Text
Watermarking for LLMs (CTWL) that allows text watermarks to carry multi-bit
customizable information. First of all, we study the taxonomy of LLM
watermarking technologies and give a mathematical formulation for CTWL.
Additionally, we provide a comprehensive evaluation system for CTWL: (1)
watermarking success rate, (2) robustness against various corruptions, (3)
coding rate of payload information, (4) encoding and decoding efficiency, (5)
impacts on the quality of the generated text. To meet the requirements of these
non-Pareto-improving metrics, we follow the most prominent vocabulary
partition-based watermarking direction, and devise an advanced CTWL method
named Balance-Marking. The core idea of our method is to use a proxy language
model to split the vocabulary into probability-balanced parts, thereby
effectively maintaining the quality of the watermarked text. Our code is
available at https://github.com/lancopku/codable-watermarking-for-llm.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要