Adaptive Crowdsourcing Via Self-Supervised Learning
CoRR(2024)
摘要
Common crowdsourcing systems average estimates of a latent quantity of
interest provided by many crowdworkers to produce a group estimate. We develop
a new approach – just-predict-others – that leverages self-supervised
learning and a novel aggregation scheme. This approach adapts weights assigned
to crowdworkers based on estimates they provided for previous quantities. When
skills vary across crowdworkers or their estimates correlate, the weighted sum
offers a more accurate group estimate than the average. Existing algorithms
such as expectation maximization can, at least in principle, produce similarly
accurate group estimates. However, their computational requirements become
onerous when complex models, such as neural networks, are required to express
relationships among crowdworkers. Just-predict-others accommodates such
complexity as well as many other practical challenges. We analyze the efficacy
of just-predict-others through theoretical and computational studies. Among
other things, we establish asymptotic optimality as the number of engagements
per crowdworker grows.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要