International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
Call for Papers | Fully Refereed | Open Access | Double Blind Peer Reviewed

ISSN: 2319-7064

ser



Downloads: 116 | Views: 208

Research Paper | Mathematics | Saudi Arabia | Volume 3 Issue 9, September 2014 | Rating: 6.6 / 10


Batch Gradient Method with Smoothing L1/2 Regularization and Momentum for Pi-sigma Networks

Kh. Sh. Mohamed [2] | Y. Sh. Mohammed [3] | A. A. Elzain | Kh. M. Makin | Elnoor. A. A. Noh [2]


Abstract: : In this paper, we study the Batch gradient method with Smoothing L1/2 Regularization and Momentum for Pi-sigma Networks, assuming that the training samples are permuted stochastically in each cycle of iteration. The usual L1/2 Regularization term involves absolute value and is not differentiable at the origin, which typically causes oscillation of the gradient method of the error function during the training. However, using the Smoothing approximation techniques, the deficiency of the norm L1/2 Regularization term can be addressed. Corresponding convergence results of Smoothing L1/2 Regularization are proved, that is, the weak convergence result is proved under the uniformly boundedness assumption of the activation function and its derivatives.


Keywords: Pi-sigma Network, Batch Gradient Method, smoothing L1/2 Regularization, Momentum, boundedness, convergence


Edition: Volume 3 Issue 9, September 2014,


Pages: 2002 - 2012


How to Download this Article?

Type Your Valid Email Address below to Receive the Article PDF Link


Verification Code will appear in 2 Seconds ... Wait

Top