MESIA: Understanding and Leveraging Supplementary Nature of Method-level Comments for Automatic Comment Generation
arxiv(2024)
摘要
Code comments are important for developers in program comprehension. In
scenarios of comprehending and reusing a method, developers expect code
comments to provide supplementary information beyond the method signature.
However, the extent of such supplementary information varies a lot in different
code comments. In this paper, we raise the awareness of the supplementary
nature of method-level comments and propose a new metric named MESIA (Mean
Supplementary Information Amount) to assess the extent of supplementary
information that a code comment can provide. With the MESIA metric, we conduct
experiments on a popular code-comment dataset and three common types of neural
approaches to generate method-level comments. Our experimental results
demonstrate the value of our proposed work with a number of findings. (1)
Small-MESIA comments occupy around 20
the WHAT comment category. (2) Being able to provide various kinds of essential
information, large-MESIA comments in the dataset are difficult for existing
neural approaches to generate. (3) We can improve the capability of existing
neural approaches to generate large-MESIA comments by reducing the proportion
of small-MESIA comments in the training set. (4) The retrained model can
generate large-MESIA comments that convey essential meaningful supplementary
information for methods in the small-MESIA test set, but will get a lower BLEU
score in evaluation. These findings indicate that with good training data,
auto-generated comments can sometimes even surpass human-written reference
comments, and having no appropriate ground truth for evaluation is an issue
that needs to be addressed by future work on automatic comment generation.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要