An Oracle-Structured Bundle Method for Distributed Optimization

We consider the problem of minimizing a function that is a sum of convex agent functions plus a convex common public function that couples them. The agent functions can only be accessed via a subgradient oracle; the public function is assumed to be structured and expressible in a domain specific language (DSL) for convex optimization. We focus on the case when the evaluation of the agent oracles can require significant effort, which justifies the use of solution methods that carry out significant computation in each iteration. We propose a cutting-plane or bundle-type method for the distributed optimization problem, which has a number of advantages over other methods that are compatible with the access methods, such as proximal subgradient methods: it has very few parameters that need to be tuned; it often produces a reasonable approximate solution in just a few tens of iterations; and it tolerates agent failures. This paper is accompanied by an open source package that implements the proposed method, available at \url{https://github.com/cvxgrp/OSBDO}.