Parallel LDA with Over-Decomposition

Latent Dirichlet Allocation (LDA) is a statistical technique for topic modeling. Prior efforts to parallelize LDA have either used expensive atomic operations or weakened the sampling model to enable parallelization without heavy use of atomics. In this paper, we present a parallel LDA implementation that uses an over-decomposed 2D tiling strategy to overcome the limitations of previous parallelization schemes. An alternate implementation that uses some approximation is also presented that fully avoids use of atomic operations. Experimental results with the NIPS and NYTimes datasets demonstrate the effectiveness of the new implementations.