[fitsbits] Bintable proposals

[William Pence]

Tom,

My inclination would be to not confuse the issue by introducing a new 8-byte
datatype at this time.  The current 'P' datatype has been widely used and
tested, so hopefully the current proposals can be swiftly approved by the
FITS committees.

Eventually I think it is reasonable to add a 'Q' data type (a pair of 8-byte
integers), but I would like more time to experiment with implementations of
it before proposing it as a standard data type.  Support for 8-byte
integers, and support for large files greater than 2**31 bytes long are both
somewhat problematic on current computer platforms.  This situation should
improve with time.  Another issue to be considered is whether they should be
interpreted as a pair of *unsigned* 8-byte integers.

-Bill Pence

Tom McGlynn wrote:
> 
> The only concern I would have with adopting these proposals is the
> restriction to 4-byte integer offsets within the variable length array
> convention.  Effectively this limits the total size of variable length records
> to about 2 GB (4 GB absolute maximum) in an extension.
> This is already marginally inadequate for the sizes of datasets
> that we have today and may be a serious bottleneck by the time the
> change gets approved.  Should we consider adding a 'Q' data type for
> binary tables which uses a pair of 8 byte integers in the same way as
> the P type uses 4 bytes now, and work to adopt this expanded recommendation?
> I think the need for 8 byte integers for file offsets is much greater than
> in data types -- and there your poll showed strong support for an eventual
> extension.
> 
> Gigapixel arrays are not too far in the future...
> 
> I'd hope this would be a relatively non-controversial change -- compared to adding
> 8 byte integers to images and binary table colums, but if it will delay things
> by years (decades!) I'd go ahead with the current proposal.
>
>> Proposal 1:
>> -------------------------------------------------------------------------
>>        Recommendation to Incorporate the "Variable Length Array"
>>           Facility in Binary Tables into the FITS Standard
>> 
>>      W.D. Pence (NASA/GSFC), D. Tody (NOAO), and W.D. Cotton (NRAO)
>>                            2 November 2001
>> 
     [ the rest deleted ]
-- 
____________________________________________________________________
Dr. William Pence                          pence at tetra.gsfc.nasa.gov
NASA/GSFC Code 662         HEASARC         +1-301-286-4599 (voice)     
Greenbelt MD 20771                         +1-301-286-1684 (fax)