[fitsbits] Adding fpzip floating point compression {External}

[Barrett, Paul]

>From the fpzip website, the lossless (fpzip) algorithm achieves compression
ratios of 1.5 - 4x depending on the data. It is also designed for streaming
with little to no latency when decompressing. The lossy (zfp) algorithm
achieves compression ratios of ~100x. The C/C++ library is freely available.

 -- Paul

On Mon, Oct 27, 2025 at 12:35 PM Dubois-Felsmann, Gregory P. <
gpdf at ipac.caltech.edu> wrote:

> Paul,
>
> The "tile" in tile compression arises, in part, from the idea that for
> quantized compression there are advantages to aligning the quantization
> with local conditions across an image.  It also facilitates accessing only
> part of a large image if all that's needed is a cutout.
>
> I'm not familiar with fpzip; can you comment on whether there are overhead
> issues that would push one toward compressing images as a whole or whether
> the "tile" approach matches up with how fpzip performs?
>
> I've found references that say that LLNL's "zfp" is designed for streaming
> processing, which would likely mean it would be well-matched to tiling as
> well.  I wasn't able to find as clear a statement on their "fpzip", but
> that might be because I'm having (temporary, I assume) trouble accessing
> the original paper.
>
> Thanks,
> Gregory
>
> ________________________________________
> From: fitsbits <fitsbits-bounces at listmgr.nrao.edu> on behalf of Seaman,
> Robert Lewis - (rseaman) via fitsbits <fitsbits at listmgr.nrao.edu>
> Sent: Monday, October 27, 2025 08:37
> To: fitsbits at listmgr.nrao.edu
> Subject: Re: [fitsbits] Adding fpzip floating point compression {External}
>
> Hi Paul,
>
> You do mean adding it as an option to FITS tile compression, right? This
> may include bintable compression, as well as image arrays.
>
> There have been a number of suggestions lately related to enabling
> additional algorithms for tile compression. It was always designed to
> support this. Any such that are adopted into FITS should (at least) be
> enabled for FPACK. It would be good if such algorithms could be evaluated
> similarly to as described in papers I and II linked at:
> https://heasarc.gsfc.nasa.gov/docs/software/fitsio/fpack/
>
> (The website is still accessible, if not being maintained, during the
> gov’t shutdown.)
>
> Speed as well as compression ratio are important in efficient data
> representations. Decompression may be faster or slower than compression,
> etc.
>
> Rob Seaman
> Lunar and Planetary Laboratory
> University of Arizona
>
> —
>
>
> On 10/27/25, 8:27 AM, "fitsbits" wrote:
>
>
> External Email
>
> ________________________________
>
> I'm not quite understanding how this would be implemented. Is this a new
> BITPIX ?   Or does it only apply to a new extension. like how BINTABLE was
> done?
>
> I get worried if we encode (compression) algorithms in order to decode
> FITS files.
>
>  - peter
>
> On 10/27/25 11:13, Barrett, Paul via fitsbits wrote:
> I would like to suggest that FITS add the fpzip<
> https://computing.llnl.gov/projects/fpzip> floating point compression
> algorithm to the FITS standard. fpzip is primarily designed for lossless
> compression but also has a provision for lossy compression.
>
> I will be adding it to FITSFiles.jl, a pure Julia implementation of the
> FITS standard.
>
>  -- Paul
>
>
> --
> Paul Barrett, PhD
> Department of Physics
> The George Washington University
> Washington, DC 20052
>
>
>
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu<mailto:fitsbits at listmgr.nrao.edu>
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/fitsbits/attachments/20251027/f0034bcf/attachment-0001.html>