Downloads: 10 | Views: 248 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Informative Article | Information Technology | United States of America | Volume 13 Issue 8, August 2024 | Popularity: 5.5 / 10
Strategic Data Pipeline Design: Enhancing Operational Efficiency from Oracle to Single Store using Airflow S3 Data Pipelines
Pankaj Dureja
Abstract: This paper investigates an innovative ETL pipeline managed by Apache Airflow, integrating Oracle databases with SingleStore through Amazon S3. The architecture enhances efficiency, scalability, and reliability of data integration processes. By implementing a sequence of orchestrated tasks, the study demonstrates improvements in data throughput and process automation compared to traditional ETL techniques. This article is significant as it addresses the need for scalable and efficient ETL processes in enterprise data management, demonstrating the potential improvements over traditional methods.
Keywords: Data Integration, ETL, Apache Airflow, Oracle Database, Oracle Functions, Oracle External Directories, SingleStore Pipelines, SingleStore Procedure, Amazon S3 Storage, Data Transformation
Edition: Volume 13 Issue 8, August 2024
Pages: 285 - 290
DOI: https://www.doi.org/10.21275/SR24731115303
Make Sure to Disable the Pop-Up Blocker of Web Browser
Similar Articles
Downloads: 1 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Research Paper, Information Technology, India, Volume 11 Issue 3, March 2022
Pages: 1642 - 1649Data Integration Strategies in Hybrid Cloud Environments
Sai Kumar Reddy Thumburu
Downloads: 1 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Research Paper, Information Technology, United States of America, Volume 13 Issue 12, December 2024
Pages: 1348 - 1357Autonomous Claims Processing: Building Self-Driving Workflows with Gen AI and ML in Guidewire
Sateesh Reddy Adavelli
Downloads: 2 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Analysis Study Research Paper, Information Technology, United States of America, Volume 11 Issue 1, January 2022
Pages: 1647 - 1652Data Engineering Challenges in AI for Healthcare
Nithin Reddy Desani, Srujan Reddy Jabbi Reddy
Downloads: 2 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Research Paper, Information Technology, India, Volume 9 Issue 12, December 2020
Pages: 1866 - 1874Data Vault 2.0: Evolution and Adoption in Large Enterprises
Guruprasad Nookala
Downloads: 3 | Weekly Hits: ⮙1 | Monthly Hits: ⮙1
Informative Article, Information Technology, India, Volume 12 Issue 12, December 2023
Pages: 2100 - 2104Bridging Data Silos: Enhancing Business Operations through Advanced Data Integration and System Interoperability
Sree Sandhya Kona