Impact of Columnar Storage Optimizations in Redshift and Big Query

Authors

  • Rameshbabu Lakshmanasamy Senior Data Engineer, Jewelers Mutual Group, USA. Author

DOI:

https://doi.org/10.47363/JEAST/2024(6)E141

Keywords:

Amazon Redshift, Google Big Query , Columnar, Compression Techniques, JSON, Parquet, ORC

Abstract

With the data explosion, companies are utilizing "new age" data warehouses, such as Amazon Redshift and Google BigQuery, to process this information. Space-efficient storage structures (specifically column-oriented storage formats) are the path to improved query performance and lower costs. Columnar storage stores the data sorted by columns and not rows so that only the columns being queried are accessed when a query is made. Fewer disk reads mean faster execution time, and that's good for analytical processing.

Author Biography

  • Rameshbabu Lakshmanasamy, Senior Data Engineer, Jewelers Mutual Group, USA.

    Senior Data Engineer, Jewelers Mutual Group, USA.

Downloads

Published

2024-03-25