Remote Proctored Exam for Proficiency-Testing: A Cross-Sectional Study

Abstract Background: The COVID-19 pandemic has profoundly affected assessment practice in higher education including a complex planning of supervision. To organise safely and reliably a remote proficiency-test for admission to the Advanced Master of General Practice (AMGP), we developed a supervisor app tracking and tracing candidates’ behaviour. Methods: A cross-sectional design was adopted with candidates applying for admission to the AMGP. The supervisor app operated on three levels to register events: recording actions, analysing behaviour, and live supervision. Each suspicious event was given a score. The outcome measures were the number of suspicious events and the exam outcome compared to the past year. To get more insight into candidates’ perceptions about the app, a post-test questionnaire was administered. An exploratory factor analysis was performed to explore quantitative data, while qualitative data were thematically analysed. Results: In total, 472 (79%) candidates used the app in an off campus setting and 121 (20%) were on campus with live supervision. Test results of both groups were comparable. The app detected 22 candidates with a suspicious level >1, mainly due to background noise. All events occurred without fraud purpose. Out of 472 candidates, 304 filled in the post-test questionnaire. Two factors were extracted from the analysis and identified as candidates’ appreciation of the app and as emotional distress because of the app. Four themes were identified in the thematic analysis providing more insight on candidates’ emotional well-being. Conclusions: A supervisor app registering and recording behaviour to prevent fraud during off-campus exams is efficient without influencing the exam outcome. Although candidates’ perceptions were mixed, increased anxiety was due to the lack of clear guidelines about the app. Future research should compare in a controlled design the cost-benefit balance between the supervisor app and candidates’ awareness of being monitored combined with a safe exam browsing plug in.