APT-Pipe: A Prompt-Tuning Tool for Social Data Annotation using ChatGPT
WWW 2024(2024)
Abstract
Recent research has highlighted the potential of LLM applications, like
ChatGPT, for performing label annotation on social computing text. However, it
is already well known that performance hinges on the quality of the input
prompts. To address this, there has been a flurry of research into prompt
tuning – techniques and guidelines that attempt to improve the quality of
prompts. Yet these largely rely on manual effort and prior knowledge of the
dataset being annotated. To address this limitation, we propose APT-Pipe, an
automated prompt-tuning pipeline. APT-Pipe aims to automatically tune prompts
to enhance ChatGPT's text classification performance on any given dataset. We
implement APT-Pipe and test it across twelve distinct text classification
datasets. We find that prompts tuned by APT-Pipe help ChatGPT achieve higher
weighted F1-score on nine out of twelve experimented datasets, with an
improvement of 7.01
a framework by showing how it can be extended to support additional tuning
mechanisms.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined