Big File Cloud Storage (BFCS): Distributed Storage Service with De-Duplication and Secure Storage

Sukruti Gajare, R. A. Khan

Abstract: Nowadays, cloud-based storage are growing and has become an emerging trend in big data storage field. Many problems arise while designing an efficient and low complicated storage engine for cloud-based systems with some issues like big files processing, metadata, latency, parallel Input/Output, de-duplication, distributed nature, high scalability. Key value stores has a vital role and showed many advantages when solving those problems. This paper presents about Big File Cloud Storage (BFCS) with its modules and architecture to handle most of problems in a big file cloud storage which is based on key value store. Here we are proposing less-complicated, fixed metadata design, which allows fast as well as highly-concurrent, distributed file Input/Output, and simple file and data de-duplication method for static data. This method can be used to build a distributed storage system that can accommodate data whose size is upto terabytes.

Keywords: Cloud Storage System, Key value, Big File, Distributed Storage System