Validating a Multiple-Choice Modelling Competencies Assessment

As part of a larger project focused on exploring development of mathematical modelling competencies among post-secondary STEM majors enrolled in advanced mathematics, we developed a pair of parallel multiple-choice modelling competencies assessments. In this chapter, we provide a technical report of item development, scale calibration, and validation of the assessment. We used multiple statistical approaches, including classical test theory (CTT), item response theory (IRT), and principal component analysis (PCA) to document item behaviours, scale properties, and dimensionality of a developing multiple-choice assessment of mathematical modelling competencies designed for post-secondary STEM majors. We share analyses and inferences, making recommendations for the field in pursuing such assessments.