64-bit integers in FITS

[Rob Seaman]

Tom McGlynn <tam at silk.gsfc.nasa.gov> writes:

> Note that while we do have ways to indicate that real data is being
> encoded as integers  (BZERO/BSCALE, ...) we do not have the inverse: that
> value encoded in a double is really an integer.  That's an important thing
> to know about an item.

Well, BSCALE and BZERO were originally designed for packing reals into
integer pixels before the floating point convention, but they now can
also mean that 16 bit integers are unsigned, so FITS has no reliable
way to indicate that the original datatype was different than the FITS
datatype in either direction.  This seems like a pretty esoteric thing
to rely on knowing in any event.

> Some things that might need 8 byte integers:
>   - As mentioned previously clock times  [...]
>   - File sizes
>   - Verification/encryption keys, e.g., an 8 byte checksum.  [...]
>   - Catalog IDs  [...]
>   - System data  [...]

However, can anybody suggest a current need for images with 64 bit
pixels?  ...and do tables and images have to support the same set of
datatypes?  "Exotic" image data arrays are likely to be written as
binary tables anyway.

On the other hand, 64 bits may not be enough for some of the applications
listed above.  Encryption keys and digital signatures, for instance, can
require hundreds or thousands of bits (and a proportionately large number
of ASCII digits or characters).  A character string may be the only thing
that suffices for table fields (or header keywords) that require extended
precision.

> With regard to Tim Pearson's question of whether we should implement
> intermediate sizes, the historical precedent seems to be no:  FITS does
> not support 24 bit ints.

Note that the hubbub about unsigned integers is also something of a
historical artifact.  If a signed integer datatype is large enough to
contain the entire range of the unsigned integers being generated (e.g.,
packing 15 bit unsigned data into 16 bit signed pixels), then there is
no problem - other than some slight overhead that may perhaps be involved
in this easy type conversion.  And if the signed datatype is not large
enough (trying to fit 18 bits into a short integer), then some higher
precision datatype is needed entirely.

The main reason for worrying about unsigned integers is that the
affordable state of the art for A/D converters is 16 bits and we don't
want to waste the high order bit.  Folks building instruments with
18 bit A/D's won't be concerned with packing them into short integers
using BZERO.

Perhaps rather than worrying about unsigned integers (or 24 bit ints or
whatever), we should get back to devising a speedy FITS compression scheme
that would allow using 32 (or even 64) bit integer pixels for intermediate
precision data in a space-efficient manner.

Rob Seaman
-- 
seaman at noao.edu, http://iraf.noao.edu/~seaman
NOAO, 950 N Cherry Ave, Tucson AZ 85719, 520-318-8248
PGP: 98 8D 8B 49 74 9A 41 88  3A 43 87 54 51 BF 30 4B