Exact Sparse Approximation Problems via Mixed-Integer Programming: Formulations and Computational Performance

Sparse approximation addresses the problem of approximately fitting a linear model with a solution having as few non-zero components as possible. While most sparse estimation algorithms rely on suboptimal formulations, this work studies the performance of exact optimization of l0-norm-based problems through Mixed-Integer Programs (MIPs). Nine different sparse optimization problems are formulated based on l1, l2 or l∞ data misfit measures, and involving whether constrained or penalized formulations. For each problem, MIP reformulations allow exact optimization, with optimality proof, for moderate-size yet difficult sparse estimation problems. Algorithmic efficiency of all formulations is evaluated on sparse deconvolution problems. This study promotes error-constrained minimization of the l0 norm as the most efficient choice when associated with l1 and l∞ misfits, while the l2 misfit is more efficiently optimized with sparsity-constrained and sparsity-penalized problems. Exact l0-norm optimization is shown to outperform classical methods in terms of solution quality, both for over- and underdetermined problems. Numerical simulations emphasize the relevance of the different lp fitting possibilities as a function of the noise statistical distribution. Such exact approaches are shown to be an efficient alternative, in moderate dimension, to classical (suboptimal) sparse approximation algorithms with l2 data misfit. They also provide an algorithmic solution to less common sparse optimization problems based on l1 and l∞ misfits. For each formulation, simulated test problems are proposed where optima have been successfully computed. Data and optimal solutions are made available as potential benchmarks for evaluating other sparse approximation methods.