Abstract of Effect of Local Dynamic Learning Rate Adaptation o
International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
Open Access | Fully Refereed | Peer Reviewed

ISSN: 2319-7064

Downloads: 149

Research Paper | Computer Science | China | Volume 9 Issue 4, April 2020

Effect of Local Dynamic Learning Rate Adaptation on Mini-Batch Gradient Descent for Training Deep Learning Models with Back Propagation Algorithm

Joyce K. Ndauka, Dr. Zheng Xiao Yan

Backpropagation has gained much popularity for training neural network models including Deep learning models. Despite its popularity, ordinary backpropagation suffers from low convergence rate due existence of constant learning rate. Since error surface is not smooth learning rate needs to be dynamically adapted to speed up rate of convergence. Much work has been done showing benefits of employing dynamic learning rate on backpropagation algorithm to speed up the rate of convergence focusing on global learning rate adaption and local adaptive learning rate on batch gradient descent. In this work we tried to observe the effect of local dynamic learning rate adaptation using improved iRprop- algorithm with mini batch gradient descent to improve convergence rate of back propagation. Experiment was conducted in python using cifar 10 dataset. Results shows the proposed algorithm outperform the ordinary back propagation algorithm in terms of speed when batch size.

Keywords: Mini-batch gradient descent, Learning rate, Backpropagation, Deep leaning

Edition: Volume 9 Issue 4, April 2020

Pages: 339 - 342

Share this Article

How to Cite this Article?

Joyce K. Ndauka, Dr. Zheng Xiao Yan, "Effect of Local Dynamic Learning Rate Adaptation on Mini-Batch Gradient Descent for Training Deep Learning Models with Back Propagation Algorithm", International Journal of Science and Research (IJSR), https://www.ijsr.net/search_index_results_paperid.php?id=SR20401112354, Volume 9 Issue 4, April 2020, 339 - 342

Enter Your Email Address


Similar Articles with Keyword 'gradient'

Downloads: 61

Research Paper, Computer Science, Sri Lanka, Volume 9 Issue 12, December 2020

Pages: 558 - 562

Artificial Neural Network Approach to Predict Milk Yield of Dairy Farm

Nayomi Jeewanthi Gamlath

Share this Article

Downloads: 106

Research Paper, Computer Science, India, Volume 2 Issue 5, May 2013

Pages: 309 - 310

Adaptive Histogram Equalization For Detecting Cancer In Digital Mammogram

Saranya Sathyamurthy, Dr. C. Lakshmi

Share this Article

Similar Articles with Keyword 'Learning'

Downloads: 169 | Weekly Hits: ⮙2 | Monthly Hits: ⮙4

Research Paper, Computer Science, United States of America, Volume 9 Issue 9, September 2020

Pages: 1095 - 1100

Maintaining Social Distancing using Artificial Intelligence

Krish Chaudhary

Share this Article

Downloads: 1 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1

Research Paper, Computer Science, India, Volume 10 Issue 9, September 2021

Pages: 656 - 661

Social Media Sentiment Analysis Using CNN-BiLSTM

Rhea Bharal, O. V. Vamsi Krishna

Share this Article

Similar Articles with Keyword 'rate'

Downloads: 169 | Weekly Hits: ⮙2 | Monthly Hits: ⮙4

Research Paper, Computer Science, United States of America, Volume 9 Issue 9, September 2020

Pages: 1095 - 1100

Maintaining Social Distancing using Artificial Intelligence

Krish Chaudhary

Share this Article

Downloads: 0

Research Paper, Computer Science, India, Volume 10 Issue 4, April 2021

Pages: 714 - 718

Character Recognition using KNN Algorithm

Sai Jahnavi Bachu

Share this Article
Top