Cross-Platform Adaptive Fault Tolerance: Bringing PDTI?s Dynamic Resilience to Apache Spark and Kubernetes

Rajani Kumari Vaddepalli

Abstract: Distributed computing frameworks like Apache Spark and Kubernetes face constant challenges in dynamic, failure-prone environments. Yet, most fault tolerance approaches remain rigid and tailored to specific platforms. Recent innovations, such as the Parallel Distributed Task Infrastructure (PDTI), have introduced adaptive fault tolerance using real-time monitoring and machine learning. However, their effectiveness across different systems is still unclear. In this paper, we explore how PDTI?s adaptive fault tolerance can be extended to major distributed frameworks like Spark and Kubernetes. We identify key architectural and algorithmic adjustments needed for smooth integration and propose a cross-platform adaptation layer. This layer retains the core advantages of dynamic failure prediction and task redistribution while adapting to each framework?s unique scheduling, communication, and recovery models. Through extensive experiments on Spark (for batch processing) and Kubernetes (for container orchestration), we assess performance, resilience, and overhead. Our results show up to 40% faster fault recovery and 15% higher throughput compared to native fault tolerance methods-without significant resource costs. These findings pave the way for universally adaptable fault tolerance in heterogeneous distributed systems, bridging the gap between specialized and general-purpose resilience solutions.

Keywords: Adaptive fault tolerance, distributed computing, cross-platform resilience, Apache Spark, Kubernetes, machine learning, failure prediction, task distribution, dynamic scheduling, heterogeneous systems, fault recovery, performance optimization

How to Cite?: Rajani Kumari Vaddepalli, "Cross-Platform Adaptive Fault Tolerance: Bringing PDTI?s Dynamic Resilience to Apache Spark and Kubernetes", Volume 11 Issue 6, June 2022, International Journal of Science and Research (IJSR), Pages: 2068-2074, https://www.ijsr.net/getabstract.php?paperid=SR22623114707, DOI: https://dx.doi.org/10.21275/SR22623114707