Next Generation of Electronic Medical Record Search Engines to Support Chart Reviews: A Systematic User Study and Future Research Direction
Journal of Economy and Technology(2024)
Abstract
Objective
Little research has been done on the user-centered document ranking approach, especially in a crowdsourcing chart review environment. As the starting point of designing and implementing the next generation of Electronic Medical Record (EMR) search engines, a systematic user study is needed to better understand the users' needs, challenges, and future research directions of EMR search engines.
Materials and Methods
One primary observation during the user study is the need for a ranking method to better support the so-called "early stopping" reviewing strategy (i.e., reviewing only a subset of EMRs of one patient to make the final decision) during the clinical chart reviews. The authors proposed two novel user-centered ranking metrics: "critical documents" and "negative guarantee ratio," to better measure the power of a ranking method in supporting the “early stopping” requirements during clinical chart reviews.
Results
The evaluation results show that i) traditional information retrieval metrics, such as the precision-at-K, have limitations in guiding the design and development of EMR search engines to better support clinical chart reviews; ii) there is not a global optimal ranking method that fits the needs of different chart reviews and different users; iii) a learning-to-rank approach cannot guarantee a stable and optimal ranking for different chart reviews and different users; and iv) A user-centered ranking metric, such as the negative guarantee ratio (NGR) metric is able to measure the “early-stopping” performance of ranking methods.
Conclusions
User-centered ranking metrics can better measure the power of ranking methods in supporting clinical chart reviews. Future research should explore more user-centered ranking metrics and evaluate their impact on real-world EMR search engines.
MoreTranslated text
Key words
electronic medical records,clinical chart reviews,search engine,ranking metrics,user study,semantic embeddings,natural language processing,information retrieval,medical context vector space
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined