[fitsbits] Proposed Changes to the FITS Standard (2)

[Francois Ochsenbein]

Hi Bill,

I'm really also concerned by this inconsistency raised by Don and I did
ask the same question before. I don't think it's wise to try to allow
heap offset to be larger than 2**31 -- if the offset is a signed integer,
then the limitation of the heap offset should be 2**31. If more is wished,
then move to 64-bit integers. I'm sure the gain in space is minimal, 
just the rare cases when the heap size is between 2**31 and 2**32 would
diminish the size of the pointers.  And 32-bit machines won't anyway be
able to use heaps larger than 2**31 anyway.

--Francois

>
>Don Wells wrote:
>> While examining the differences document,
>[at http://fits.gsfc.nasa.gov/fits_draft.html]
>> I encountered two sentences in
>> 7.3.5 that seem to me to be inconsistent.  The first is "The meaning of
>> a negative value for either of these integers [in the array descriptor]
>> is not defned by this standard."  The second is "The storage referenced
>> by an array descriptor must lie entirely within the heap area; negative
>> offsets are not permitted." The first sentence leaves open the option of
>> a negative offset (and that is an obvious escape hatch, because the
>> integers are definitely *signed*), but the second sentence closes off
>> the option.
>
>Back in 2005 when the IAU FITS Working Group approved the variable
>length array convention as part of the FITS Standard, it was decided to
>specifically define that the 2 integers in the "P" array descriptor are
>"signed" 32-bit integers (and not "unsigned" integers) and to add the
>statement that "The meaning of a negative value for either of these
>integers is not defned by this standard."
>
>Allowing negative array descriptor values is not necessarily a
>contradiction to the later statement that "negative offsets [in the
>heap] are not permitted", because it is possible to define a convention
>that maps (via some mathematical function) a negative descriptor value
>into a positive heap offset value.  Note that the same thing was done
>long ago with the BITPIX keyword; originally the BITPIX value had to be
>positive, but later the definition was extended so that if the value is
>negative then the actual number of bits per pixel is derived by
>multiplying the BITPIX value by -1.
>
>Arguably, the most useful function for mapping a negative array
>descriptor value into a positive heap offset is this one:
>
>      heap_offset = 2**32 + descriptor_value
>                  = 4294967296 + descriptor_value
>
>(where the descriptor_value is a negative signed 32 bit integer in the
>range -1 to -2147483648).  This function effectively doubles the useful
>size of the heap:  positive descriptor values can refer to heap offsets
>in the range 0 to 2147483647, and the negative descriptor values map
>into heap offsets in the range 2147483648 to 4294967295.
>
>Incidently, it is no coincidence that this mapping function produces
>exactly the same positive heap offset value as would be given if one
>were to interpret the 32-bit descriptor value as an unsigned 32-bit
>integer.
>
>This begs the question of what, if anything, should be done to clarify
>the use or legality of negative array descriptor values in variable
>length array columns in binary tables.  Should the mapping function
>described above be viewed as a blatant attempt to circumvent the
>restriction on interpreting the array descriptors as "signed" integers
>and thus should be expressly forbidden in the Standard, or, on the other
>hand, is this a clever adaptation to work within the rules of FITS to
>allow new functionality (by doubling the useful size of the heap area)?
>    My on feeling on this is that we should either leave the wording of
>the Standard as it is (with the intentional ambiguity about the meaning
>of negative array descriptor values), or, go back to the fundamental
>issue and redefine the array descriptors to be unsigned integers, which
>by definition would eliminate the need to explain the interpretation of
>negative descriptor values.
>
>Bill Pence
>--
>____________________________________________________________________
>Dr. William Pence                       pence at milkyway.gsfc.nasa.gov
>NASA/GSFC Code 662       HEASARC        +1-301-286-4599 (voice)
>Greenbelt MD 20771                      +1-301-286-1684 (fax)
>
>
>_______________________________________________
>fitsbits mailing list
>fitsbits at listmgr.cv.nrao.edu
>http://listmgr.cv.nrao.edu/mailman/listinfo/fitsbits

================================================================================
Francois Ochsenbein       ------       Observatoire Astronomique de Strasbourg
   11, rue de l'Universite F-67000 STRASBOURG       Phone: +33-(0)390 24 24 29
Email: francois at astro.u-strasbg.fr   (France)         Fax: +33-(0)390 24 24 32
================================================================================