Downloads: 116 | Views: 148
Research Paper | Mathematics | Saudi Arabia | Volume 3 Issue 9, September 2014
Batch Gradient Method with Smoothing L1/2 Regularization and Momentum for Pi-sigma Networks
Kh. Sh. Mohamed  | Y. Sh. Mohammed  | A. A. Elzain | Kh. M. Makin | Elnoor. A. A. Noh 
Abstract: : In this paper, we study the Batch gradient method with Smoothing L1/2 Regularization and Momentum for Pi-sigma Networks, assuming that the training samples are permuted stochastically in each cycle of iteration. The usual L1/2 Regularization term involves absolute value and is not differentiable at the origin, which typically causes oscillation of the gradient method of the error function during the training. However, using the Smoothing approximation techniques, the deficiency of the norm L1/2 Regularization term can be addressed. Corresponding convergence results of Smoothing L1/2 Regularization are proved, that is, the weak convergence result is proved under the uniformly boundedness assumption of the activation function and its derivatives.
Keywords: Pi-sigma Network, Batch Gradient Method, smoothing L1/2 Regularization, Momentum, boundedness, convergence
Edition: Volume 3 Issue 9, September 2014,
Pages: 2002 - 2012
Similar Articles with Keyword 'Momentum'
Downloads: 1 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Research Paper, Mathematics, Kenya, Volume 10 Issue 11, November 2021Pages: 523 - 529
Analysis of Turbulent Natural Convection with Localized Heating on the Ceiling and on the Floor and Cooling on Opposite Vertical Walls in a Rectangular Enclosure
Omariba G. Ong'era | Johana K. Sigey  | Jeconia A. Okelo  | Stephen M. Karanja
Research Paper, Mathematics, Kenya, Volume 10 Issue 9, September 2021Pages: 530 - 534
A Numerical Study of Turbulent Natural Convection in a Rectangular Enclosure with Localised Heating and Cooling
Clementine Kaluki Mutua | Kennedy Awuor | Emily Nafula John