[fitsbits] reopening of Public Comment Period on the compression conventions

[William Thompson]

On 01/13/16 13:30, William Pence wrote:
> When using algorithms like PLIO or Rice, which are designed to efficiently
> compress data masks containing a pixel array of ones and zeros, it makes no
> difference what integer data type one uses to represent the mask.  The
> compressed masks will have exactly the same size, regardless of the original
> mask data type. This is because these algorithms effectively just record the
> location of the non-zero pixels, so how one represents those non-zero pixels (as
> a bit, byte, or longer integer word) has no effect.
>
> The main reason that support for the NULL_PIXEL_MASK option has not (yet) been
> implemented in CFITSIO is that there are not many use cases where this
> capability is needed.  Given the current set of supported compression
> algorithms, the only time it might be useful is when using the H-Compress
> algorithm in it's lossy compression mode to achieve a high degree of compression
> of integer images (typically with compression ratios of 10 to 100).  However,
> these highly compressed images are typically only used to provide a 'quick look'
> preview of the image, and it would be ill-advised to try to do any quantitative
> analysis on such images.  Therefore, there is not much need to exactly preserve
> the location of any null pixels in such highly degraded images.

The use of lossy compression schemes like H-compress and ICER with compression 
ratios as high as 40 have been used routinely to bring down science data from 
solar missions, because of telemetry bandwidth limitations.  Admittedly, this is 
distinct from compressing the data again for storage once it's on the ground, 
but such high compression ratios don't necessarily preclude quantitative analysis.

Compression ratios for quicklook images can be as high as 800, though that would 
also include binning as part of the compression.

Bill Thompson

-- 
William Thompson
NASA Goddard Space Flight Center
Code 671
Greenbelt, MD  20771
USA

301-286-2040
William.T.Thompson at nasa.gov