Downloads: 116 | Views: 148
Research Paper | Mathematics | Saudi Arabia | Volume 3 Issue 9, September 2014
Batch Gradient Method with Smoothing L1/2 Regularization and Momentum for Pi-sigma Networks
Kh. Sh. Mohamed [2] | Y. Sh. Mohammed [3] | A. A. Elzain | Kh. M. Makin | Elnoor. A. A. Noh [2]
Abstract: : In this paper, we study the Batch gradient method with Smoothing L1/2 Regularization and Momentum for Pi-sigma Networks, assuming that the training samples are permuted stochastically in each cycle of iteration. The usual L1/2 Regularization term involves absolute value and is not differentiable at the origin, which typically causes oscillation of the gradient method of the error function during the training. However, using the Smoothing approximation techniques, the deficiency of the norm L1/2 Regularization term can be addressed. Corresponding convergence results of Smoothing L1/2 Regularization are proved, that is, the weak convergence result is proved under the uniformly boundedness assumption of the activation function and its derivatives.
Keywords: Pi-sigma Network, Batch Gradient Method, smoothing L1/2 Regularization, Momentum, boundedness, convergence
Edition: Volume 3 Issue 9, September 2014,
Pages: 2002 - 2012
Similar Articles with Keyword 'Momentum'
Downloads: 1 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Research Paper, Mathematics, Kenya, Volume 10 Issue 11, November 2021
Pages: 523 - 529Analysis of Turbulent Natural Convection with Localized Heating on the Ceiling and on the Floor and Cooling on Opposite Vertical Walls in a Rectangular Enclosure
Omariba G. Ong'era | Johana K. Sigey [6] | Jeconia A. Okelo [3] | Stephen M. Karanja
Downloads: 2
Research Paper, Mathematics, Kenya, Volume 10 Issue 9, September 2021
Pages: 530 - 534A Numerical Study of Turbulent Natural Convection in a Rectangular Enclosure with Localised Heating and Cooling
Clementine Kaluki Mutua | Kennedy Awuor | Emily Nafula John