Assessing the Performance of Chinese Open Source Large Language Models in Information Extraction Tasks
arxiv(2024)
Abstract
Information Extraction (IE) plays a crucial role in Natural Language
Processing (NLP) by extracting structured information from unstructured text,
thereby facilitating seamless integration with various real-world applications
that rely on structured data. Despite its significance, recent experiments
focusing on English IE tasks have shed light on the challenges faced by Large
Language Models (LLMs) in achieving optimal performance, particularly in
sub-tasks like Named Entity Recognition (NER). In this paper, we delve into a
comprehensive investigation of the performance of mainstream Chinese
open-source LLMs in tackling IE tasks, specifically under zero-shot conditions
where the models are not fine-tuned for specific tasks. Additionally, we
present the outcomes of several few-shot experiments to further gauge the
capability of these models. Moreover, our study includes a comparative analysis
between these open-source LLMs and ChatGPT, a widely recognized language model,
on IE performance. Through meticulous experimentation and analysis, we aim to
provide insights into the strengths, limitations, and potential enhancements of
existing Chinese open-source LLMs in the domain of Information Extraction
within the context of NLP.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined