Optimizing interacting Langevin dynamics using spectral gaps

Using independent runs of an optimization algorithm is a standard scheme for finding the minimizer of an objective function. In this paper we seek to improve this practice by allowing the different optimizers to interact. The question that arises is: how can one choose the interaction structure that will result in the fastest convergence while maintaining minimal communication costs? To investigate this issue we formulate it through the optimization of the spectral gap of interacting Langevin dynamics. In the case of a linear interaction, the spectral gap is directly related to the spectrum of the Laplacian matrix that characterizes the interaction. We present early numerical results in both convex and non-convex settings that illustrate the benefit of choosing the right kind of interaction structure.