Tight Bounds for ℓ1 Oblivious Subspace Embeddings

AbstractAn \(\ell _p\) oblivious subspace embedding is a distribution over \(r \times n\) matrices \(\Pi\) such that for any fixed \(n \times d\) matrix A, \[ \Pr _{\Pi }[\textrm {for all }x, \ \Vert Ax\Vert _p \le \Vert \Pi Ax\Vert _p \le \kappa \Vert Ax\Vert _p] \ge 9/10, \] where r is the dimension of the embedding, \(\kappa\) is the distortion of the embedding, and for an n-dimensional vector y, \(\Vert y\Vert _p = (\sum _{i=1}^n |y_i|^p)^{1/p}\) is the \(\ell _p\)-norm. Another important property is the sparsity of \(\Pi\), that is, the maximum number of non-zero entries per column, as this determines the running time of computing \(\Pi A\). While for \(p = 2\) there are nearly optimal tradeoffs in terms of the dimension, distortion, and sparsity, for the important case of \(1 \le p \lt 2\), much less was known. In this article, we obtain nearly optimal tradeoffs for \(\ell _1\) oblivious subspace embeddings, as well as new tradeoffs for \(1 \lt p \lt 2\). Our main results are as follows:(1)We show for every \(1 \le p \lt 2\), any oblivious subspace embedding with dimension r has distortion\[ \kappa = \Omega \left(\frac{1}{\left(\frac{1}{d}\right)^{1 / p} \log ^{2 / p}r + \left(\frac{r}{n}\right)^{1 / p - 1 / 2}}\right). \]When \(r = {\operatorname{poly}}(d) \ll n\) in applications, this gives a \(\kappa = \Omega (d^{1/p}\log ^{-2/p} d)\) lower bound, and shows the oblivious subspace embedding of Sohler and Woodruff (STOC, 2011) for \(p = 1\) is optimal up to \({\operatorname{poly}}(\log (d))\) factors.(2)We give sparse oblivious subspace embeddings for every \(1 \le p \lt 2\). Importantly, for \(p = 1\), we achieve \(r = O(d \log d)\), \(\kappa = O(d \log d)\) and \(s = O(\log d)\) non-zero entries per column. The best previous construction with \(s \le {\operatorname{poly}}(\log d)\) is due to Woodruff and Zhang (COLT, 2013), giving \(\kappa = \Omega (d^2 {\operatorname{poly}}(\log d))\) or \(\kappa = \Omega (d^{3/2} \sqrt {\log n} \cdot {\operatorname{poly}}(\log d))\) and \(r \ge d \cdot {\operatorname{poly}}(\log d)\); in contrast our \(r = O(d \log d)\) and \(\kappa = O(d \log d)\) are optimal up to \({\operatorname{poly}}(\log (d))\) factors even for dense matrices. We also give (1) \(\ell _p\) oblivious subspace embeddings with an expected \(1+\varepsilon\) number of non-zero entries per column for arbitrarily small \(\varepsilon \gt 0\), and (2) the first oblivious subspace embeddings for \(1 \le p \lt 2\) with \(O(1)\)-distortion and dimension independent of n. Oblivious subspace embeddings are crucial for distributed and streaming environments, as well as entrywise \(\ell _p\) low-rank approximation. Our results give improved algorithms for these applications.