Downloads: 117 | Views: 188
Research Paper | Computer Science & Engineering | India | Volume 3 Issue 11, November 2014
Lightning Fast Distributed Machine Learning Framework
Abstract: According to IBM, Big Data can be expressed with 4 Vs, namely Volume, Velocity, Variety and Veracity. Lots of companies are incorporating Big Data in their business model to derive insights from the unstructured data. Big Data is analyzed using Statistical Methods and Machine Learning. Limiting factor using traditional technologies is the incompetence to use huge amounts of data to learn or train algorithms within a practical time. This problem can be handled by using in-memory and distributed machine learning techniques with the help of distributed data sets and by allocating learning to various workstations. A distributed machine learning framework is developed with Spark, Hadoop and Python to scale the machine learning algorithm and to reduce the intensive computation.
Keywords: Big Data, Distributed Machine Learning, Apache Spark, Python
Edition: Volume 3 Issue 11, November 2014,
Pages: 2913 - 2915