Algorithm/architecture solutions to improve beyond uniform quantization in embedded DNN accelerators

The choice of data type has a major impact on speed, accuracy, and power consumption of deep learning accelerators. Quantizing the weights and activations of neural networks to integer based computation is an industry standard for reducing memory footprint and computation cost of inference in embedded systems. Uniform weight quantization can be used for tasks where accuracy drop can be tolerated. However, the drop in accuracy due to a uniform quantization might be non-negligible especially when performed on shallow networks, complex computer vision tasks, or with lower-bit integers.