International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
Call for Papers | Fully Refereed | Open Access | Double Blind Peer Reviewed

ISSN: 2319-7064

Downloads: 2 | Views: 26 | Weekly Hits: ⮙2 | Monthly Hits: ⮙2

Informative Article | Data & Knowledge Engineering | India | Volume 7 Issue 11, November 2018

Design and Implementation of High-Throughput Data Streams using Apache Kafka for Real-Time Data Pipelines

Preyaa Atri [3]

Abstract: In an era dominated by the need for real-time data processing, Apache Kafka emerges as a crucial technology for constructing high-throughput data pipelines capable of handling extensive data streams with minimal latency. This paper provides an in-depth exploration into the design and implementation of Kafka-based data pipelines, discussing their architectural patterns, performance optimization techniques, and practical applications across various domains. Through detailed analysis and expert recommendations, this study addresses current challenges and maps out future research directions, underlining Kafka's pivotal role in advancing real-time data processing systems. The insights presented aim to guide professionals in enhancing the efficiency and reliability of their real-time data solutions.

Keywords: Apache Kafka, data streams, real-time processing, data pipelines, high- throughput, distributed systems, stream processing, big data

Edition: Volume 7 Issue 11, November 2018,

Pages: 1988 - 1991

How to Download this Article?

Type Your Valid Email Address below to Receive the Article PDF Link

Verification Code will appear in 2 Seconds ... Wait