Apache Parquet is an open-source column-oriented data storage format that supports large data sets because it’s optimised for disk I/O and can achieve high compression ratios with columnar data.
Beeks Analytics' high performance decoding layer can capture, decode and egress up to 300 million messages per second per appliance; for even higher volume requirements, we can stream using Apache Arrow or transfer the data using the Parquet data format, both of which allow you to reach even higher message rates.