On Tight Approximate Inference of the Logistic-Normal Topic Admixture Model

The Logistic-Normal Topic Admixture Model (LoNTAM), also known as correlated topic model (Blei and Lafierty, 2005), is a promis- ing and expressive admixture-based text model. It can capture topic correlations via the use of a logistic-normal distribu- tion to model non-trivial variabilities in the topic mixing vectors underlying documents. However, the non-conjugacy caused by the logistic-normal makes posterior inference and model learning signiflcantly more challeng- ing. In this paper, we present a new, tight ap- proximate inference algorithm for LoNTAM based on a multivariate quadratic Taylor ap- proximation scheme that facilitates elegant closed-form message passing. We present ex- perimental results on simulated data as well as on the NIPS17 and PNAS document col- lections, and show that our approach is not only simple and easy to implement, but also it converges faster, and leads to more accu- rate recovery of the semantic truth underly- ing documents and estimates of the parame- ters comparing to previous methods.