[fitsbits] Adding fpzip floating point compression {External}

[Dubois-Felsmann, Gregory P.]

Paul, 

The "tile" in tile compression arises, in part, from the idea that for quantized compression there are advantages to aligning the quantization with local conditions across an image.  It also facilitates accessing only part of a large image if all that's needed is a cutout.

I'm not familiar with fpzip; can you comment on whether there are overhead issues that would push one toward compressing images as a whole or whether the "tile" approach matches up with how fpzip performs?

I've found references that say that LLNL's "zfp" is designed for streaming processing, which would likely mean it would be well-matched to tiling as well.  I wasn't able to find as clear a statement on their "fpzip", but that might be because I'm having (temporary, I assume) trouble accessing the original paper.

Thanks,
Gregory

________________________________________
From: fitsbits <fitsbits-bounces at listmgr.nrao.edu> on behalf of Seaman, Robert Lewis - (rseaman) via fitsbits <fitsbits at listmgr.nrao.edu>
Sent: Monday, October 27, 2025 08:37
To: fitsbits at listmgr.nrao.edu
Subject: Re: [fitsbits] Adding fpzip floating point compression {External}

Hi Paul,

You do mean adding it as an option to FITS tile compression, right? This may include bintable compression, as well as image arrays.

There have been a number of suggestions lately related to enabling additional algorithms for tile compression. It was always designed to support this. Any such that are adopted into FITS should (at least) be enabled for FPACK. It would be good if such algorithms could be evaluated similarly to as described in papers I and II linked at: https://heasarc.gsfc.nasa.gov/docs/software/fitsio/fpack/

(The website is still accessible, if not being maintained, during the gov’t shutdown.)

Speed as well as compression ratio are important in efficient data representations. Decompression may be faster or slower than compression, etc.

Rob Seaman
Lunar and Planetary Laboratory
University of Arizona

—


On 10/27/25, 8:27 AM, "fitsbits" wrote:


External Email

________________________________

I'm not quite understanding how this would be implemented. Is this a new BITPIX ?   Or does it only apply to a new extension. like how BINTABLE was done?

I get worried if we encode (compression) algorithms in order to decode FITS files.

 - peter

On 10/27/25 11:13, Barrett, Paul via fitsbits wrote:
I would like to suggest that FITS add the fpzip<https://computing.llnl.gov/projects/fpzip> floating point compression algorithm to the FITS standard. fpzip is primarily designed for lossless compression but also has a provision for lossy compression.

I will be adding it to FITSFiles.jl, a pure Julia implementation of the FITS standard.

 -- Paul


--
Paul Barrett, PhD
Department of Physics
The George Washington University
Washington, DC 20052



_______________________________________________
fitsbits mailing list
fitsbits at listmgr.nrao.edu<mailto:fitsbits at listmgr.nrao.edu>
https://listmgr.nrao.edu/mailman/listinfo/fitsbits