[fitsbits] Error in the FITS Standard document {External}

[Richard Shaw]

Dear Bill,

I concur with your suggested revision to Sect. 10.1.3.

Regards,
Dick

On Fri, Oct 27, 2023 at 12:01 PM <fitsbits-request at listmgr.nrao.edu> wrote:

> Send fitsbits mailing list submissions to
>         fitsbits at listmgr.nrao.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>
> You can reach the person managing the list at
>         fitsbits-owner at listmgr.nrao.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of fitsbits digest..."
>
>
> Today's Topics:
>
>    1. Error in the FITS Standard document  {External} (William Pence)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Thu, 26 Oct 2023 13:55:12 -0400
> From: William Pence <wdpence2000 at yahoo.com>
> To: FITSbits <fitsbits at nrao.edu>
> Subject: [fitsbits] Error in the FITS Standard document  {External}
> Message-ID: <37349B71-BFAF-4157-8CA8-74D5DAB7010E at yahoo.com>
> Content-Type: text/plain; charset=utf-8
>
> ?Andreas Korpi-Lagg (MPI) and some of the astropy developers recently
> alerted me to an apparent incorrect statement in the current FITS Standard
> document. This occurs in section 10.1.3 of that document in the discussion
> about compressed arrays:
>
> ?The compressed data columns described above may use either the ?1P? or
> ?1Q? variable-length array FITS column format if the size of the heap in
> the compressed FITS file is < 2.1 GB. If the heap is larger than 2.1 GB,
> then the ?1Q? format (which uses 64-bit pointers) must be used.?
>
> That second sentence is not correct  because it is possible to create a
> legal FITS file where the heap is larger than 2.1 GB using the ?1P? column
> format, for instance if the sum of the 2 pointer values in the P descriptor
> is greater than 2^31. One could also create a FITS file with an arbitrarily
> large heap containing extra space that is not used by the P variable-length
> array column.
>
> My suggestion is to replace those 2 sentences in the FITS Standard with
> the following text:
>
> ?The compressed data columns described above may use either the ?1P? or
> ?1Q? variable-length array FITS column format if the 2 integers in the
> descriptor have values that are within the range supported by the 32-bit
> pointers in the ?1P? column format.  Otherwise, the ?1Q? format which uses
> 64-bit pointers must be used.?
>
> FWIW, I would recommend always using the 1Q format for variable length
> arrays.  Then you don?t have to worry about whether the array is too large
> to fit, and the savings in file size by using the 1P format is usually
> negligible.
>
> Bill Pence
> NASA/HEASARC Emeritus
>
>
>
>
>
>
>
>
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>
>
> ------------------------------
>
> End of fitsbits Digest, Vol 148, Issue 2
> ****************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/fitsbits/attachments/20231112/f6ab30e0/attachment.html>