Chrome Extension
WeChat Mini Program
Use on ChatGLM

Nonexpert Crowds Outperform Expert Individuals in Diagnostic Accuracy on a Skin Lesion Diagnosis Task.

ISBI(2023)

Cited 0|Views22
No score
Abstract
A recent study [1] showed that individual physicians with at least ten years of experience as dermatologists achieved 74.7% accuracy on average in labeling images from the multiclass International Skin Imaging Collaboration (ISIC) 2018 challenge dataset. Using a novel gamified crowdsourcing method, we collected 144,383 nonexpert opinions over two weeks on the medical image annotation platform DiagnosUs, and the resulting crowd consensus labels obtained by aggregating using a plurality rule achieved a significantly higher accuracy of 78.1% (p=0.0014), a multiclass ROC AUC (area under the receiver operating characteristic curve) of 0.948 (95% CI 0.936-0.959), and malignant versus benign ROC AUC of 0.928 (95% CI 0.911-0.943). These results suggest an opportunity to harness gamified methods to assist in the creation of high-quality labeled datasets that could benefit medical artificial intelligence (AI) development.
More
Translated text
Key words
Skin lesion classification,ISIC,crowdsourcing,gamification
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined