[fitsbits] 64-bit integers

[William Pence]

On Wed, May 11, 2005 at 12:12:02PM -1000, Thierry Forveille wrote:
> William Pence writes:
>  > 3.  A couple of months ago, CFITSIO was modified to demonstrate the 
>  > feasibility of supporting unsigned integer descriptors.  This turned out to 
>  > be trivial to implement.  It provides a practical demonstration that there 
>  > is no difficulty in supporting the larger heap size.  Since there was never 
>  > any practical use for a negative signed descriptor value, it seems 
>  > desirable to reverse the decision made 6-months ago and now define the 
>  > descriptors to be unsigned integers, thus doubling the maximum size of 
>  > the heap.  This change cannot affect or invalidate any existing FITS 
>  > files so it does not violate the 'once FITS always FITS rule'.
>  > 
> On the other hand, introducing unsigned integers specififically for
> heap pointers slightly mars the homogeneity of the standard, since they
> are not supported in any other role. Also, 2^63 gets us to 9 million
> Terabytes, so even if disk capacity keep doubling at its current
> every ~18 months it will take another 30+ years before the difference
> between 2^64 and 2^63 could possibly matter.

Indeed, at a rate of 1 GB/s it would take 292 years to write 2^63 bytes of data!  I 
agree that there is no need to worry about an extra factor of 2 for the 64-bit
descriptors.  In the case of the 32-bit 'P' descriptor, however, I think the difference
between  4.2 GB or 2.1 GB for the max heap size will have real practical consequences.
In some cases the extra factor of 2 could mean an application could continue to use 
32-bit 'P' pointers instead of having to move up to 64-bit 'Q' pointers.

I might add that after adding support for 64-bit 'Q' descriptor columns 
to CFITSIO, I was pleasantly surprised to realize how deeply the descriptor 
size (32-bit or 64-bit) is hidden from any application program.
The descriptor address is only used internally within cfitsio, so any application
program that can read or write 'P' variable length array columns will be able
to read and write 'Q' columns, with absolutely no change.  I expect the same will
be true for other FITS I/O interfaces besides CFITSIO.  Since these descriptor
values are so well hidden, I don't think there is any reason to not use unsigned
integers to gain the extra factor of 2 in size.  This is quite different
from the signed integer values in 'I', 'J', and 'K' columns which are directly
visible to applications programs.  

Bill Pence