Development of a formative assessment instrument to determine students' need for corrective actions in physics: Identifying students' functional level of understanding

In physics education, most teachers provide students feedback on their problem solutions through grades on written tests. The practice of feedback after a summative test does not often meet the needs of many students to improve their problem solving. In this paper we report on the development of a formative assessment instrument to allow teachers to provide more meaningful action-oriented feedback on students' performance on written tests. Our research and development approach comprised three phases.The first phase consisted of a literature guided cognitive analysis of effective problem-solving strategies in the physics domain. This analysis resulted in the identification of three crucial episodes in students' problem-solving approaches during which students engage in specific cognitive activities. The second phase consisted of the design of an assessment instrument to monitor specific cognitive activities during the three crucial episodes when solving physics problems. This resulted in a rating scale with 11 levels to indicate students' efficacy. The third phase consisted of research of the validity, reliability, and practicality of the instrument. Here we trained three teachers to trace students' mistakes on different problems in the domain of kinematics and asked them to rate the mastery of 16 eleventh-grade pre-university students. In this phase we assessed the reliability and validity of this instrument by computing Krippendorff's alpha to indicate teachers' inter-rater reliability. Practicality of the instrument was assessed by examining the variation in students' level of mastery on problems of different complexity. Further research is needed to provide more detailed guidelines for how teachers can use the instrument in formative assessment (in contrast to summative assessment) to help students to develop correct solution methods and foster students' metacognition about problem solving in related physics areas (i.e., knowledge transfer).