[fitsbits] FITS 'P' descriptors: signed or unsigned?

[William Pence]

Preben Grosbol wrote:
> On Thursday 16 June 2005 00:21, William Pence wrote:
> 
>>At issue is whether to reverse the recent decision to define the 'P'
>>variable-length array descriptors in FITS binary tables to be a pair of
>>'signed 32-bit integers', and make them 'unsigned 32-bit integers' instead.
> 
> I'm sorry but we cannot 'reverse the recent decision'.  That would violate
> the section 9 of the FITS Standard 'Restrictions on Changes'

With respect, changing the descriptors from 'signed' to 'unsigned' will not 
violate section 9 (which states "Any structure that is a valid FITS 
structure shall remain a valid FITS structure at all future times".)  Any 
existing FITS file that contains positive signed integer heap descriptors 
(i.e., using only 31 bits) will continue to be a valid FITS file if the 
recent decision is reversed to define the pointers as unsigned integers 
(allowing full use of all 32 bits).

In fact, it was the previous decision (to make the descriptors 'signed 
integers') that violated Section 9.  Up until April 2005 it was legal to 
make FITS files with descriptor values greater than 2GB, but now they are 
invalid.  Even though we made a deliberate decision to overlook the Section 
9 requirement in that case (on the grounds that we did not know of any 
existing FITS files that had heaps larger than 2GB), it could be argued that 
the previous  decision was 'unconstitutional', and therefore we must restore 
the less restrictive requirement that the descriptors be interpreted as 
unsigned integers.

My own view on whether the 'P' pointers should be signed or unsigned 
integers is mainly driven by what what would be most useful for the FITS 
user community.  Currently CFITSIO (and probably other FITS libraries as 
well) supports binary tables with heaps up to 4.2 GB in size.  Why should we 
  imposing an artificial restriction that only allows heaps that are 1/2 as 
large?

In addition, I can see almost no software development cost to supporting 
these larger heaps.  Any FITS file that makes use of this extended pointer 
range will necessarily be greater than 2GB in size (which in itself is 
perfectly legal since there has never been any restriction on the size of a 
FITS file).  Any software that accesses these Large files must necessarily 
be using 64-bit addressing (at least on most computer platforms) to perform 
seeks and offsets in the file.  Since the software is already handling 
64-bit addressing, there should be no difficulty in supporting the full 
range of heap addresses allowed by the unsigned 32-bit descriptor.   In any 
case, it is only the small number of FITS interface libraries that need to 
be modified to support the unsigned pointers.  This should be completely 
transparent to any software that calls the FITS library to read and write 
the arrays in the heap, so existing applications programs should not need to 
be modified to support the larger heap size.

Bill Pence
-- 
____________________________________________________________________
Dr. William Pence                          William.D.Pence at nasa.gov
NASA/GSFC Code 662         HEASARC         +1-301-286-4599 (voice)
Greenbelt MD 20771                         +1-301-286-1684 (fax)