Untangling Lariats: Subgradient Following of Variationally Penalized Objectives

We describe a novel subgradient following apparatus for calculating the optimum of convex problems with variational penalties. In this setting, we receive a sequence y_i,…,y_n and seek a smooth sequence x_1,…,x_n. The smooth sequence attains the minimum Bregman divergence to an input sequence with additive variational penalties in the general form of ∑_i g_i(x_i+1-x_i). We derive, as special cases of our apparatus, known algorithms for the fused lasso and isotonic regression. Our approach also facilitates new variational penalties such as non-smooth barrier functions. We next derive and analyze multivariate problems in which 𝐱_i,𝐲_i∈ℝ^d and variational penalties that depend on 𝐱_i+1-𝐱_i. The norms we consider are ℓ_2 and ℓ_∞ which promote group sparsity. Last but not least, we derive a lattice-based subgradient following for variational penalties characterized through the output of arbitrary convolutional filters. This paradigm yields efficient solvers for problems in which sparse high-order discrete derivatives such as acceleration and jerk are desirable.