Scaling up Machine Learning Algorithms for Large Datasets

Manju Joy

Abstract: Machine Learning is the field of study that gives computers the ability to learn without being explicitly programmed. There is a need to explore techniques for scaling up learning algorithms so that it can be applied to problems with millions of training examples, thousands of features, and hundreds of classes. Traditionally, the bottleneck preventing the development of more-intelligent systems via machine learning was limited data available. However, in many domains, the size of the datasets available now is so large and powerful learning algorithms are needed to learn from infinite data in finite time. This paper is a review of works in machine learning on methods for handling data sets containing large amounts of information. A method is proposed to handle this problem which is based on K-means clustering.

Keywords: Relevant features, Feature selection, Decision tree, Clustering