LLMClean: Context-Aware Tabular Data Cleaning via LLM-Generated OFDs
CoRR(2024)
Abstract
Machine learning's influence is expanding rapidly, now integral to
decision-making processes from corporate strategy to the advancements in
Industry 4.0. The efficacy of Artificial Intelligence broadly hinges on the
caliber of data used during its training phase; optimal performance is tied to
exceptional data quality. Data cleaning tools, particularly those that exploit
functional dependencies within ontological frameworks or context models, are
instrumental in augmenting data quality. Nevertheless, crafting these context
models is a demanding task, both in terms of resources and expertise, often
necessitating specialized knowledge from domain experts.
In light of these challenges, this paper introduces an innovative approach,
called LLMClean, for the automated generation of context models, utilizing
Large Language Models to analyze and understand various datasets. LLMClean
encompasses a sequence of actions, starting with categorizing the dataset,
extracting or mapping relevant models, and ultimately synthesizing the context
model. To demonstrate its potential, we have developed and tested a prototype
that applies our approach to three distinct datasets from the Internet of
Things, healthcare, and Industry 4.0 sectors. The results of our evaluation
indicate that our automated approach can achieve data cleaning efficacy
comparable with that of context models crafted by human experts.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined