Real-Time Stream Processing of Big Data

Denis Patrick Bell, Eliasu Tambominyi, Yang Chunting

Abstract: The evolution of technology has enabled the continuous generation of massive data from connected devices and sensors. As, more data becomes available, organizations are using cutting-edge tools and techniques to extract useful insight from the data immediately they are generated. Therefore, the enterprise systems are evolving and shifting towards real-time systems. So, there is a growing need for well-developed fault tolerant distributed scalable systems to handle this change with low latency. Recent years have seen the emergence of several distributed systems whose popularity is as a result of the growing demand to efficiently analyze and interpret voluminous data. This article will discuss distributed real-time stream processing of big data, the two main real-time big data processing architectures: Lambda and Kappa and the popular frameworks used for processing real time big data.

Keywords: Real-time data processing, Big data, Batch processing, Lambda & Kappa architecture

How to Cite?: Denis Patrick Bell, Eliasu Tambominyi, Yang Chunting, "Real-Time Stream Processing of Big Data", Volume 10 Issue 3, March 2021, International Journal of Science and Research (IJSR), Pages: 1247-1252, https://www.ijsr.net/getabstract.php?paperid=SR21320045639, DOI: https://dx.doi.org/10.21275/SR21320045639