Exploring the True Potential: Evaluating the Black-box Optimization Capability of Large Language Models
arxiv(2024)
摘要
Large language models (LLMs) have gained widespread popularity and
demonstrated exceptional performance not only in natural language processing
(NLP) tasks but also in non-linguistic domains. Their potential as artificial
general intelligence extends beyond NLP, showcasing promising capabilities in
diverse optimization scenarios. Despite this rising trend, whether the
integration of LLMs into these black-box optimization problems is genuinely
beneficial remains unexplored. This paper endeavors to tackle this issue by
offering deeper insights into the potential of LLMs in optimization tasks
through a comprehensive investigation. Our approach involves a comprehensive
evaluation, covering both discrete and continuous optimization problems, aiming
to assess the efficacy and distinctive characteristics that LLMs bring to the
realm of optimization. Our findings reveal both the limitations and advantages
of LLMs in optimization. On one hand, despite consuming the significant power
required to run the model, LLMs exhibit subpar performance and lack desirable
properties in pure numerical tasks, primarily due to a mismatch between the
problem domain and their processing capabilities. On the other hand, although
LLMs may not be ideal for traditional numerical optimization, their potential
in broader optimization contexts remains promising. LLMs exhibit the ability to
solve problems in non-numerical domains and can leverage heuristics from the
prompt to enhance their performance. To the best of our knowledge, this work
presents the first systematic evaluation of LLMs for numerical optimization,
offering a progressive, wide-coverage, and behavioral analysis. Our findings
pave the way for a deeper understanding of LLMs' role in optimization and guide
future application in diverse scenarios for LLMs.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要