LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agents
CoRR(2024)
摘要
Large language models (LLMs) have recently received considerable attention as
alternative solutions for task planning. However, comparing the performance of
language-oriented task planners becomes difficult, and there exists a dearth of
detailed exploration regarding the effects of various factors such as
pre-trained model selection and prompt construction. To address this, we
propose a benchmark system for automatically quantifying performance of task
planning for home-service embodied agents. Task planners are tested on two
pairs of datasets and simulators: 1) ALFRED and AI2-THOR, 2) an extension of
Watch-And-Help and VirtualHome. Using the proposed benchmark system, we perform
extensive experiments with LLMs and prompts, and explore several enhancements
of the baseline planner. We expect that the proposed benchmark tool would
accelerate the development of language-oriented task planners.
更多查看译文
关键词
task planning,language models,benchmarking,embodied agents,home robots
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要