LUNA: A Framework for Language Understanding and Naturalness Assessment
CoRR(2024)
Abstract
The evaluation of Natural Language Generation (NLG) models has gained
increased attention, urging the development of metrics that evaluate various
aspects of generated text. LUNA addresses this challenge by introducing a
unified interface for 20 NLG evaluation metrics. These metrics are categorized
based on their reference-dependence and the type of text representation they
employ, from string-based n-gram overlap to the utilization of static
embeddings and pre-trained language models.
The straightforward design of LUNA allows for easy extension with novel
metrics, requiring just a few lines of code. LUNA offers a user-friendly tool
for evaluating generated texts.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined