ALPBench: A Benchmark for Active Learning Pipelines on Tabular Data
CoRR(2024)
摘要
In settings where only a budgeted amount of labeled data can be afforded,
active learning seeks to devise query strategies for selecting the most
informative data points to be labeled, aiming to enhance learning algorithms'
efficiency and performance. Numerous such query strategies have been proposed
and compared in the active learning literature. However, the community still
lacks standardized benchmarks for comparing the performance of different query
strategies. This particularly holds for the combination of query strategies
with different learning algorithms into active learning pipelines and examining
the impact of the learning algorithm choice. To close this gap, we propose
ALPBench, which facilitates the specification, execution, and performance
monitoring of active learning pipelines. It has built-in measures to ensure
evaluations are done reproducibly, saving exact dataset splits and
hyperparameter settings of used algorithms. In total, ALPBench consists of 86
real-world tabular classification datasets and 5 active learning settings,
yielding 430 active learning problems. To demonstrate its usefulness and broad
compatibility with various learning algorithms and query strategies, we conduct
an exemplary study evaluating 9 query strategies paired with 8 learning
algorithms in 2 different settings. We provide ALPBench here:
https://github.com/ValentinMargraf/ActiveLearningPipelines.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要