International Journal of Science and Research (IJSR)

International Journal of Science and Research (IJSR)
Since Year 2012 | Open Access | Double Blind Reviewed

ISSN: 2319-7064




Downloads: 108

Research Paper | Computer Science & Engineering | India | Volume 4 Issue 7, July 2015


Parallel Data Shuffling for Hadoop Acceleration with Network Levitated Merge and RDMA for Interconnectivity

Kishorkumar Shinde | Venkatesan N.


Abstract: Performance is measure issue in todays hadoop framework. The execution time required for Map reduce model is depends on multiple factors. Shuffling and merging in map reduce requires much amount of time. Proper implementation of shuffling and merging improves the performance of overall system. With this Serialization, multiple interconnect issues are also covered in this paper. Serialization keeps reduce phase to wait, repetitive merges requires multiple disk access and lack of portability for different interconnections. Repetitive merges can be reduced by network levitated merge algorithm, Serialization issue is overcome by parallelization. RDMA is used to for multiple interconnects. A non Hadoop and non java machine can also use the hadoop features. If we use pipelining to avoid serialization some sort of serialization is there in shuffle and merge phase. In pipelining output file is shuffled and merged before providing it to reduce task. Instead of pipelined shuffling, parallel shuffling is proposed. This reduces the number of disk accesses resulting in improved performance.


Keywords: Hadoop, Network levitated merge, MapReduce, Big- data, RDMA


Edition: Volume 4 Issue 7, July 2015,


Pages: 1096 - 1101


How to Cite this Article?

Kishorkumar Shinde, Venkatesan N., "Parallel Data Shuffling for Hadoop Acceleration with Network Levitated Merge and RDMA for Interconnectivity", International Journal of Science and Research (IJSR), Volume 4 Issue 7, July 2015, pp. 1096-1101, https://www.ijsr.net/get_abstract.php?paper_id=SUB156536

How to Share this Article?

Enter Your Email Address




Similar Articles with Keyword 'Hadoop'

Downloads: 1

Research Paper, Computer Science & Engineering, India, Volume 10 Issue 6, June 2021

Pages: 1188 - 1193

Profit Contribution of Bank Customer from Different Business Liabilities

Vinod Desai | Shalini B Ullagaddi | Vittal A Odeyar

Share this Article

Downloads: 1

Research Paper, Computer Science & Engineering, India, Volume 11 Issue 1, January 2022

Pages: 1229 - 1231

Big Data in Healthcare

Pratiksha Patil

Share this Article


Top