FITS long integer support (was Re: [fitsbits] ADASS FITS BoF on Sunday)

[Thomas McGlynn]

Clive Page wrote:

> On Tue, 19 Oct 2004, William Pence wrote:
> 
> 
>>Please let me know if you have ideas for other topics (regardless of
>>whether you plan to attend the meeting).
> 
> 
> I shall not be at ADASS, unfortunately, but one topic raised on this list
> recently is of interest to me: whether we can overcome the limits implied
> by 32-bit integers.
> 
> With the help of Bill and his colleagues at GSFC I was able to build a
> version of cFITSIO which handled files of over 2 GB, but I think a 2
> gigarow limit in tables may be enforced by the presence of implicitly
> 32-bit signed integers for NAXIS2 (etc.) in the headers.  It would be
> interesting to know whether any other people have an interest in extending
> FITS beyond the 32-bit limit, and whether they have ideas to help solve
> it.
> 
> Regards
> 

Hi Clive,

There are two questions here that can get confused.  I'm not
sure which you are interested in.

I don't think there is any implication that header keywords are limited
to what is permitted in 4 byte integers, i.e.,

    SIMPLE  =  T
    BITPIX  =  8
    NAXIS   =  2
    NAXIS1  =  10000000000
    NAXIS2  =  2
    END

is perfectly legal for describing a 20 GB file.   There isn't a problem
in the FITS standard with this, but some (many?) FITS readers will
have trouble doing anything with this data.  This doesn't have anything
to do with longwords directly but typically with internal limits
in compilers and operating systems.

There are also three changes to the FITS standard that would be needed
to accommodate long integers.

    BITPIX  = 64

would indicate arrays of 8 byte integers in images.

    TFORMxx = 'K'

would indicate arrays of 8 byte integers in tables.

    TFORMxx = 'Q'

would indicate use of longwords in pointers in variable length columns.

The first two are already supported in a number of FITS readers (CFITSIO I think plus
at least one IDL and one Java implementation at least).  I don't know that any implementation
support longs in variable length arrays.  Currently the only legal way to write 8 byte
integers is in ASCII tables where integers of arbitrary size can be written -- though
I doubt many readers can read these without losing precision.


So is the question that we want to improve the quality of FITS readers (and writers)
so that they can read and write files that are already legal, or do we wish
to extend the standard to accommodate a new type?

	Regards,
	Tom