Detecting Careless Respondents in Multidimensional Forced Choice Data: An Application of l z Person-Fit Statistic to the TIRT Model

There has been growing interest in the use of multidimensional forced-choice (MFC) measures in industrial and organizational psychology. However, despite the growing popularity, there has been a lack of research on how to address the problem of careless responses and improve the quality of MFC data. This study aims to address this gap by discussing the issue of careless responses in MFC data, proposing a method for detecting careless respondents using the l z person-fit statistic in conjunction with the Thurstonian item response theory (TIRT; Brown & Maydeu-Olivares, 2011) model, and examining the performance of this method on pair and triplet MFC measures and its impact on psychometric properties. To this end, we conducted empirical and simulation studies under various experimental conditions. We also provided an R tutorial for applying the l z person-fit statistic to both pair and triplet MFC measures using the TIRT model. The results showed that our proposed method can effectively detect careless respondents without a significant risk of misclassification. The implications of the findings and practical guidelines were discussed.