Designing a Resilient Parallel Distributed Task Infrastructure for Large-Scale Data Processing

Mahidhar Mullapudi, Mahesh Babu Munjala, Chinmay Kulkarni

Abstract: This technical paper introduces a sophisticated Parallel Distributed Task Infrastructure (PDTI) meticulously crafted to address the escalating demands of large-scale data processing[1][2] and transformation systems. PDTI orchestrates parallel tasks across distributed nodes, strategically optimizing latency, throughput, and processing power. The infrastructure's core components include a parent task overseeing task distribution, progress tracking, and error management across child tasks, with a paramount focus on resilience and fault tolerance. This paper expounds upon the architectural intricacies and contractual guidelines of PDTI, presenting a holistic solution to efficiently navigate the challenges inherent in processing extensive datasets within distributed computing environments[3][4][5][6].

Keywords: Parallel Distributed Task, Scalable data processing, Resilient Computing, Task Synchronization

How to Cite?: Mahidhar Mullapudi, Mahesh Babu Munjala, Chinmay Kulkarni, "Designing a Resilient Parallel Distributed Task Infrastructure for Large-Scale Data Processing", Volume 9 Issue 6, June 2020, International Journal of Science and Research (IJSR), Pages: 1925-1929, https://www.ijsr.net/getabstract.php?paperid=SR24203233945, DOI: https://dx.doi.org/10.21275/SR24203233945