Validity of Force Concept Inventory evaluated by students' explanations and confirmation using modified item response curve

In this study, we evaluate the validity of question 29 (Q.29) in the Force Concept Inventory (FCI), which requires an understanding of atmospheric pressure. At least in some Japanese physics classes, the rate of the correct answer to Q.29 is reduced between the pretest and post-test. To examine this deterioration of a student's response, we analyzed 429 explanations written by 358 respondents about the force described in the question. Many learners, who were not aware of atmospheric force, chose the correct answer. On the other hand, many of the wrong respondents had learned about air pressure but did not understand it. As a result, postinstruction learners tended to have a worse rate of correct response. This made Q.29 less sensitive to a Japanese student's ability. This characteristic feature of Q.29 is visually confirmed by a modified item response curve, which represents correlation between the response to one item and the total score of the other items.