One Size Does Not Fit All: Customizing Open-Domain Procedures.
CoRR(2023)
摘要
How-to procedures, such as how to plant a garden, are ubiquitous. But one
size does not fit all - humans often need to customize these procedural plans
according to their specific needs, e.g., planting a garden without pesticides.
While LLMs can fluently generate generic procedures, we present the first study
on how well LLMs can customize open-domain procedures. We introduce
CustomPlans, a probe dataset of customization hints that encodes diverse user
needs for open-domain How-to procedures. Using LLMs as CustomizationAgent and
ExecutionAgent in different settings, we establish their abilities to perform
open-domain procedure customization. Human evaluation shows that using these
agents in a Sequential setting is the best, but they are good enough only ~51%
of the time. Error analysis shows that LLMs do not sufficiently address user
customization needs in their generated procedures.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要