[fitsbits] FITS RICE decompression feedback {External}

[Pierre Fernique]

Dear Fits mailing list members,

We (CDS and more precisely myself) ave encountered a few difficulties in 
implementing in java the RICE decompression codes described in the 
latest version of the FITS standard 
<https://fits.gsfc.nasa.gov/standard40/fits_standard40aa-le.pdf> in the 
context of updating our Hierarchical Progressive Survey (IVOA Standard 
HiPS 1.0 <https://www.ivoa.net/documents/HiPS/>) generation tools, and 
in particular Hipsgen <https://aladin.u-strasbg.fr/hips/#tools>.

We realised that the images we were decompressing were not strictly 
identical to those decompressed by cfitsio-based tools. After 
investigation, it seems that the discrepancy was due to a variation in 
the way the ZERO_VALUE value was taken into account. In the Fits 4.0 
document, the value used to encode the zero in the SUBTRACTIVE DITHER 2 
method is 214748364*7* (page 47 "... /pixels in the floating-point image 
that are exactly equal to 0.0 are represented by the reserved value 
-2147483647 in the quantized integer array.../"). Except if I'm wrong, 
in the cfitsio library code, the value used is -214748364*6* 
(imcompress.c 
<https://heasarc.gsfc.nasa.gov/FTP/software/fitsio/c/imcompress.c> line 
10 & 7679). One of the consequences of this divergence is the shift in 
the application of the random function (inhibited for the value 
ZERO_VALUE). And indeed, having adopted the constant used by cfitsio, we 
obtain the same results.

During this exercise, we also noted that the random function required 
for compression and decompression in the SUBTRACTIVE_DITHER method (1 
and 2) and provided in Appendix I of the Fits 4.0 document is expressly 
stated as "not normative" ("/This appendix is not part of the FITS 
Standard, but is included for informational purposes/"). This point may 
mean that some FITS compressors could use another random function. And 
If this is the case, we wondered how such a variation could be detected.

At this stage, we decided to strictly follow the choices of the latest 
version of cfitsio (4.2.0)  (214748364*6 *and assuming the unique random 
function).

If this feedback is indeed relevant, perhaps the Fits 4.0 document could 
be clarified on these two points to avoid any potential divergence.

Best regards
Pierre Fernique
Centre de Données astronomiques de Strasbourg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/fitsbits/attachments/20230613/9dc94273/attachment.html>