The Emergence of Large Language Models in Static Analysis: A First Look through Micro-Benchmarks
CoRR(2024)
摘要
The application of Large Language Models (LLMs) in software engineering,
particularly in static analysis tasks, represents a paradigm shift in the
field. In this paper, we investigate the role that current LLMs can play in
improving callgraph analysis and type inference for Python programs. Using the
PyCG, HeaderGen, and TypeEvalPy micro-benchmarks, we evaluate 26 LLMs,
including OpenAI's GPT series and open-source models such as LLaMA. Our study
reveals that LLMs show promising results in type inference, demonstrating
higher accuracy than traditional methods, yet they exhibit limitations in
callgraph analysis. This contrast emphasizes the need for specialized
fine-tuning of LLMs to better suit specific static analysis tasks. Our findings
provide a foundation for further research towards integrating LLMs for static
analysis tasks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要