Peer Grading Eliciting Truthfulness Based on Autograder
IEEE Transactions on Learning Technologies(2023)
Abstract
Peer grading has diverse applications in many fields, including the peer grading of open assignments in online courses. The major challenge in peer grading is improving the seriousness (reviewing carefully) of reviewers. Previous studies have proposed several incentive reward mechanisms intended to reward or punish reviewers. Although these mechanisms are effective in massive open online courses environments with a large number of students, they are not suitable for small private online courses (SPOCs) environments with a small number of students. This article analyzes the characteristics of reviewers in a small private online course environment and designs an incentive and reward mechanism based on a hybrid human-machine peer grading framework, including an autograder (reviewed by the machine). The main component of the proposed mechanism is the autograder whose function is to evaluate the reviewers' reports (reviewers refer to students) and provide feedback information on reports to reviewers. The framework can incentivize reviewers to review carefully and give their reports truthfully when the reports between reviewers' and the autograder' are consistent. The proposed framework is verified experimentally. The experimental results demonstrate that the hybrid human-machine peer grading framework can effectively incentivize reviewers to review submissions carefully and improve the overall effect of peer grading.
MoreTranslated text
Key words
Autograder,eliciting truthfulness,hybrid human-machine,peer grading,small private online courses (SPOCs)
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined