Worth the effort? A practical examination of random effects in hidden Markov models for animal telemetry data

1. Hidden Markov models (HMMs) that include individual-level random effects have recently been promoted for inferring animal movement behaviour from biotelemetry data. These mixed HMMs come at significant cost in terms of implementation and computation, and discrete random effects have been advocated as a practical alternative to more computationally intensive continuous random effects. However, the performance of mixed HMMs has not yet been sufficiently explored to justify their widespread adoption, and there is currently little guidance for practitioners weighing the costs and benefits of mixed HMMs for a particular research objective. 2. I performed an extensive simulation study comparing the performance of a suite of fixed and random effect models for individual heterogeneity in the hidden state process of a two-state HMM. I focused on sampling scenarios more typical of telemetry studies, which often consist of relatively long time series (30-250 observations per animal) for relatively few individuals (5-100 animals). 3. I generally found mixed HMMs did not improve state assignment relative to standard HMMs. Reliable estimation of random effects required larger sample sizes than are often feasible in telemetry studies. Continuous random effect models performed reasonably well with data generated under discrete random effects, but not vice versa. Random effects accounting for unexplained individual variation can improve estimation of state transition probabilities and measurable covariate effects, but discrete random effects can be a relatively poor (and potentially misleading) approximation for continuous variation. 4. When weighing the costs and benefits of mixed HMMs, three important considerations are study objectives, sample size and model complexity. HMM applications often focus on state assignment with little emphasis on heterogeneity in state transition probabilities, in which case random effects in the hidden state process simply may not be worth the additional effort. However, if explaining variation in state transition probabilities is a primary objective and sufficient explanatory covariates are not available, then random effects are worth pursuing as a more parsimonious alternative to individual fixed effects. 5. To help put my findings in context and illustrate some potential challenges that practitioners may encounter when applying mixed HMMs, I revisit a previous analysis of long-finned pilot whale biotelemetry data.