Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based Study
CoRR(2024)
摘要
Large Language Models (LLMs) have gained significant attention in the
software engineering community. Nowadays developers have the possibility to
exploit these models through industrial-grade tools providing a handy interface
toward LLMs, such as OpenAI's ChatGPT. While the potential of LLMs in assisting
developers across several tasks has been documented in the literature, there is
a lack of empirical evidence mapping the actual usage of LLMs in software
projects. In this work, we aim at filling such a gap. First, we mine 1,501
commits, pull requests (PRs), and issues from open-source projects by matching
regular expressions likely to indicate the usage of ChatGPT to accomplish the
task. Then, we manually analyze these instances, discarding false positives
(i.e., instances in which ChatGPT was mentioned but not actually used) and
categorizing the task automated in the 467 true positive instances (165
commits, 159 PRs, 143 issues). This resulted in a taxonomy of 45 tasks which
developers automate via ChatGPT. The taxonomy, accompanied with representative
examples, provides (i) developers with valuable insights on how to exploit LLMs
in their workflow and (ii) researchers with a clear overview of tasks that,
according to developers, could benefit from automated solutions.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要