Dynamic Coding for Distributed Matrix Multiplication

Matrix multiplication is a fundamental operation in many machine learning algorithms. With the size of the dataset increasing rapidly, it is now a common practice to compute large-scale matrix multiplication on multiple servers, such that each server multiplies submatrices inside the input matrices. As straggling servers are inevitable in a distributed infrastructure, various coding schemes have been proposed which deploy coded tasks encoded from the submatrices of input matrices. The overall result can then be decoded from a subset of such coded tasks. However, as the resources are shared with other jobs in a distributed infrastructure and their performance can change dynamically, the optimal way to encode the input matrices may also change over time. So far, existing coding schemes for matrix multiplication all require to split the input matrices and encode them in advance, and cannot change the coding schemes or adjust their parameters after encoding. In this paper, we propose a coding framework that can dynamically change the coding schemes and their parameters, by only re-encoding local data in each task. We demonstrate that the original tasks can be quickly converted into new tasks only incurring marginal overhead.