Large scale manifold transduction

We show how the regularizer of Transduc- tive Support Vector Machines (TSVM) can be trained by stochastic gradient descent for linear models and multi-layer architectures. The resulting methods can be trained on- line, have vastly superior training and test- ing speed to existing TSVM algorithms, can encode prior knowledge in the network archi- tecture, and obtain competitive error rates. We then go on to propose a natural gen- eralization of the TSVM loss function that takes into account neighborhood and mani- fold information directly, unifying the two- stage Low Density Separation method into a single criterion, and leading to state-of-the- art results.