Cleaner Pretraining Corpus Curation with Neural Web Scraping
CoRR(2024)
摘要
The web contains large-scale, diverse, and abundant information to satisfy
the information-seeking needs of humans. Through meticulous data collection,
preprocessing, and curation, webpages can be used as a fundamental data
resource for language model pretraining. However, when confronted with the
progressively revolutionized and intricate nature of webpages,
rule-based/feature-based web scrapers are becoming increasingly inadequate.
This paper presents a simple, fast, and effective Neural web Scraper
(NeuScraper) to help extract primary and clean text contents from webpages.
Experimental results show that NeuScraper surpasses the baseline scrapers by
achieving more than a 20
extracting higher-quality data to facilitate the language model pretraining.
All of the code is available at https://github.com/OpenMatch/NeuScraper.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要