Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks
arxiv(2024)
摘要
We introduce a new family of prompt injection attacks, termed Neural Exec.
Unlike known attacks that rely on handcrafted strings (e.g., "Ignore previous
instructions and..."), we show that it is possible to conceptualize the
creation of execution triggers as a differentiable search problem and use
learning-based methods to autonomously generate them.
Our results demonstrate that a motivated adversary can forge triggers that
are not only drastically more effective than current handcrafted ones but also
exhibit inherent flexibility in shape, properties, and functionality. In this
direction, we show that an attacker can design and generate Neural Execs
capable of persisting through multi-stage preprocessing pipelines, such as in
the case of Retrieval-Augmented Generation (RAG)-based applications. More
critically, our findings show that attackers can produce triggers that deviate
markedly in form and shape from any known attack, sidestepping existing
blacklist-based detection and sanitation approaches.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要