标题：Training Multilayer Perceptron By Using Optimal Input Normalization
作者：Cai, Xun; Tyagi, Kanishka; Manry, Michael T.
作者机构：[Cai, Xun] Shandong Univ, Sch Comp Sci & Technol, Jinan 250100, Shandong, Peoples R China.; [Tyagi, Kanishka; Manry, Michael T.] Univ Texas Arlingto 更多
会议名称：IEEE International Conference on Fuzzy Systems (FUZZ 2011)
会议日期：JUN 27-30, 2011
来源：IEEE INTERNATIONAL CONFERENCE ON FUZZY SYSTEMS (FUZZ 2011)
关键词：multilayer perceptron (MLP); orthogonal least square (OLS);; Gauss-Newton; output weights optimization back propagation (OWO-BP);; optimal learning factor (OLF); hidden weights optimization (HWO);; Schmidt procedure
摘要：In this paper, we propose a novel second order paradigm called optimal input normalization (OIN) to solve the problems of slow convergence and high complexity of MLP. By optimizing the non-orthogonal transformation matrix of input units in an equivalent network, OIN absorbs separate optimal learning factor for each synaptic weight as well as the threshold of hidden unit, leading to an improvement in the performance for MLP training. Moreover, by using a whitening transformation of negative Jacobian matrix of hidden weights, a modified version of OIN called optimal input normalization with hidden weights optimization (OIN-HWO) is also proposed. The Hessian matrices in both OIN and OIN-HWO are computed by using Gauss-Newton method. All the linear equations are solved via orthogonal least square (OLS). Regression simulations are performed on several real-life datasets and the results show that the proposed OIN has not only much better convergence rate and generalization ability than output weights optimization-back propagation (OWO-BP), optimal input gains (OIG) and even Levenberg-Marquardt (LM) method, but also takes less computational time than OWO-BP. Although OIN-HWO takes a little expensive computational burden than OIN, its convergence rate is faster than OIN and often close to or rivals LM. It is therefore suggested that OIN-based algorithms are potentially very good choices for practical applications.