International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
Call for Papers | Fully Refereed | Open Access | Double Blind Peer Reviewed

ISSN: 2319-7064

Downloads: 2 | Views: 55 | Weekly Hits: ⮙2 | Monthly Hits: ⮙2

Informative Article | Engineering Science | India | Volume 12 Issue 10, October 2023 | Rating: 4.7 / 10

Evaluating and Selecting Appropriate Technologies and Tools for Data Ingestion: Apache Kafka and Custom - Built Solutions

Fasihuddin Mirza [6]

Abstract: The landscape of data ingestion technologies is rapidly evolving to meet the growing demands of big data analytics. As organizations strive to gain real - time insights from their data, the need for robust, scalable, and efficient data ingestion platforms has never been more critical. This paper presents a comprehensive evaluation of Apache Kafka, a prominent open - source streaming platform, against custom - built data ingestion solutions. We examine the scalability, performance, fault tolerance, and ease of integration of these technologies. A systematic approach is employed to evaluate their suitability for various use cases, including high - throughput systems, IoT environments, and real - time analytics.

Keywords: Apache Kafka, Data ingestion technologies, Stream processing, Big data analytics, Real - time data processing, Scalability, Fault tolerance, Performance benchmarking, System integration, Custom - built data solutions, Messaging systems, Distributed systems, Internet of Things (IoT), High - throughput systems, Data pipeline architecture, Publish - subscribe model, Data integrity, Data volume, Cloud computing, Data engineering.

Edition: Volume 12 Issue 10, October 2023,

Pages: 2140 - 2143

How to Download this Article?

Type Your Valid Email Address below to Receive the Article PDF Link

Verification Code will appear in 2 Seconds ... Wait