Split Questionnaire Designs for Online Surveys: The Impact of Module Construction on Imputation Quality

Established face-to-face surveys encounter increasing pressures to move online. Such a mode switch is accompanied with methodological challenges, including the need to shorten the questionnaire that each respondent receives. Split Questionnaire Designs (SQDs) randomly assign respondents to different fractions of the full questionnaire (modules) and, subsequently, impute the data that are missing by design. Thereby, SQDs reduce the questionnaire length for each respondent. Although some researchers have studied the theoretical implications of SQDs, we still know little about their performance with real data, especially regarding potential approaches to constructing questionnaire modules. In a Monte Carlo study with real survey data, we simulate SQDs in three module-building approaches: random, same topic, and diverse topics. We find that SQDs introduce bias and variability in univariate and especially in bivariate distributions, particularly when modules are constructed with items of the same topic. However, single topic modules yield better estimates for correlations between variables of the same topic.