Hierarchical Autoencoder-based Lossy Compression for Large-scale High-resolution Scientific Data
arxiv(2023)
摘要
Lossy compression has become an important technique to reduce data size in
many domains. This type of compression is especially valuable for large-scale
scientific data, whose size ranges up to several petabytes. Although
Autoencoder-based models have been successfully leveraged to compress images
and videos, such neural networks have not widely gained attention in the
scientific data domain. Our work presents a neural network that not only
significantly compresses large-scale scientific data, but also maintains high
reconstruction quality. The proposed model is tested with scientific benchmark
data available publicly and applied to a large-scale high-resolution climate
modeling data set. Our model achieves a compression ratio of 140 on several
benchmark data sets without compromising the reconstruction quality. 2D
simulation data from the High-Resolution Community Earth System Model (CESM)
Version 1.3 over 500 years are also being compressed with a compression ratio
of 200 while the reconstruction error is negligible for scientific analysis.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要