Exploring Novice Programmer Testing Behavior: A First Step to Define Coding Struggle

To promote good coding practices, we need to understand what students do when they are on their own. In this research study, we explore students' testing behavior and response to persistent errors to better understand their coding patterns. We investigate how those patterns change when they struggle, and how help-seeking might influence their coding behaviors. We define struggle during coding as failing the same unit test case consecutively for more than four submission events, considering only unit test cases created by the instructors. To analyze the students' coding data, we use progress indicators, student test implementation indicators, and both student-generated and instructor-generated unit test results from each student submission event. In addition, we use office hours attendance records and amount of assignment-related posts created on the course forum. Results show that students tend not to follow test-driven development practices, even when explicitly directed to, and tend to create unit tests only to earn assignment credit rather than to guide their software development. Students also tend not to modify their own unit tests once they have earned the related credits, even when facing coding struggle; they tend to modify their unit tests only after they have been facing coding struggle for an extended number of submission events.