Hadoop Ecosystem: An Introduction

Sneha Mehta, Viral Mehta

Abstract: Hadoop a de facto industry standard has become kernel of the distributed operating system for Big data. Hadoop has gained its popularity due to its ability of storing, analyzing and accessing large amount of data, quickly and cost effectively through clusters of commodity hardware. But, No one uses kernel alone. Hadoop is taken to be a combination of HDFS and MapReduce. To complement the Hadoop modules there are also a variety of other projects that provide specialized services and are broadly used to make Hadoop laymen accessible and more usable, collectively known as Hadoop Ecosystem. All the components of the Hadoop ecosystem, as explicit entities are evident to address particular needs. Recent Hadoop ecosystem consists of different level layers, each layer performing different kind of tasks like storing your data, processing stored data, resource allocating and supporting different programming languages to develop various applications in Hadoop ecosystem

Keywords: HDFS, HBase, MapReduce, YARN, Hive, Pig, Mahout, Avro, Sqoop, Oozie, Chukwa, Flume, Zookeeper