A Novel Pooling Block for Improving Lightweight Deep Neural Networks

•A bypass structure, Pooling Block, is proposed to fuse feature from different layers.•It’s easy to add Pooling Block to lightweight networks with a few extra parameters.•Experiments are conducted on 7 widely used datasets to validate Pooling Block.•Experimental results show improvements with Pooling Block for many CV tasks.•Pooling Block can accelerates the speed of convergence for image classification.