Parallel Algorithms for Exact Enumeration of Deep Neural Network Activation Regions
arxiv(2024)
摘要
A feedforward neural network using rectified linear units constructs a
mapping from inputs to outputs by partitioning its input space into a set of
convex regions where points within a region share a single affine
transformation. In order to understand how neural networks work, when and why
they fail, and how they compare to biological intelligence, we need to
understand the organization and formation of these regions. Step one is to
design and implement algorithms for exact region enumeration in networks beyond
toy examples.
In this work, we present parallel algorithms for exact enumeration in deep
(and shallow) neural networks. Our work has three main contributions: (1) we
present a novel algorithm framework and parallel algorithms for region
enumeration; (2) we implement one of our algorithms on a variety of network
architectures and experimentally show how the number of regions dictates
runtime; and (3) we show, using our algorithm's output, how the dimension of a
region's affine transformation impacts further partitioning of the region by
deeper layers.
To our knowledge, we run our implemented algorithm on networks larger than
all of the networks used in the existing region enumeration literature.
Further, we experimentally demonstrate the importance of parallelism for region
enumeration of any reasonably sized network.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要