Dynamic Non-Parametric Mixture Models and The Recurrent Chinese Restaurant Process

Dirichlet process mixture models provide a flexible Bayesian framework for density estimation; however they are inadequate with respect to modeling sequential data due to the full exchangeability assumption they employ. In this paper we present the temporal Dirichlet process mixture model (TDPM) as a framework for modeling complex longitudinal data. In a TDPM, the data is divided into epochs; all data points inside the same epoch are fully exchangeable, whereas the temporal order is maintained across epochs. Moreover, The number of mixture components in each epoch is unbounded: the components can retain, die out or emerge over time, and the actual parameterization of each component can also evolve over time in a Markovian fashion. We give three equivalent construction of this process as well as a Gibbs sampling algorithm to carry out posterior inference. We demonstrate our model by using it to build an infinite dynamic mixture of Gaussian factors, and a simple non-parametric dynamic topic model applied to the NIPS12 collection.