Performance Enhancing Analysis for Data in Motion in Big Data

Abstract

Big data systems are growing in size from terabytes to petabytes and zetabytes, and they have features that are structured, semi-structured, and unstructured. As a result, ensuring the safety of the data while it is being stored and moved is currently the most important requirement for businesses. Organizations like NIST are now working to define the architecture framework for big data security because security is still a relatively new concept in the big data business world. Within the context of a big data security framework, this paper explores the need, challenges, and potential solutions for protecting the privacy and integrity of data in transit. Protocols that are specialized to Big Data, such as Hadoop RPC and HDFS, as well as standard network protocols, such as Kerberos, SSL, and TLS, can work together to ensure the secure movement of large amounts of data. With the help of the AES cryptographic algorithm and its several variants, this study begins a collaborative system for the transfer of the contents of massive data. This study delves into the AES cryptographic system and its potential for performance optimization in Big Data settings, with the aim of achieving outstanding outcomes. It is proposed here that a structure consisting of five steps, which may be implemented in large data systems such as Hadoop, is a structure that focuses on efficiency and performance criteria.