标题：Rethinking the role of activation functions in deep convolutional neural networks for image classification
作者：Zheng, Qinghe ;Yang, Mingqiang ;Tian, Xinyu ;Wang, Xiaochen ;Wang, Deqiang
作者机构：[Zheng, Qinghe ;Yang, Mingqiang ;Wang, Deqiang ] School of Information Science and Engineering, Shandong University, Jimo, Qingdao; Shandong; 266237, 更多
关键词：Activation function; Deep learning; Generalization; Image classification; Overfitting
摘要：Deep convolutional neural network used for image classification is an important part of deep learning and has great significance in the field of computer vision. Moreover, it helps humans to simulate the human brain more realistically, pointing out the direction for the development of artificial intelligence. In fact, the rapid development and its application of deep neural networks are due to the improvements of various activation functions. The activation function is one of the most critical parts of the neural networks, which provides the possibility of strong nonlinear fitting ability of the deep neural network. In this paper, we analyze how the activation function affects the deep neural network, and then analyzes and summarizes the development status and the performance of different activation functions. Based on these, we designed a new activation function to improve the classification performance of neural networks. Finally, we perform extensive classification experiments on the MNIST, CIFAR10/100, and ImageNet datasets, and compare various popular activation functions to provide a reference for the selection of activation functions when designing deep neural network models. Deep convolutional neural networks, including the four models AlexNet, VGGNet, GoogLeNet, and Network in Network (NIN), are used to observe the role of the activation function in training and testing phase. The experimental results show that the constructed deep convolutional neural networks based on the improved activation function not only have a faster convergence rate, but also can improve the image classification accuracy more effectively. © 2020, International Association of Engineers. All rights reserved.