[fitsbits] Bintable proposals

[Mark Calabretta]

On Wed 2001/11/14 23:32:11 CDT, Don Wells wrote
in a message to: FITSBITS <fitsbits at nrao.edu>

>The main purpose is to enable a reader to skip over FITS extensions
>quickly.  A secondary motivation is to enable a reader to pre-allocate
>space for data it has not yet encountered.

Dear Don,

Accepting what you and Bill say about its history (while noting that the
same limitation applies to random groups, which predate extensions, and is
what got us into RPFITS in the first place), these are the only reasons
given so far:

   (1) skip over extensions quickly.

   (2) preallocate disk space,

Preallocating disk space is irrelevant if you're writing to tape!  Anyway,
is it really necessary these days?  Skipping over extensions *quickly* is
nice, but not compelling, especially if you've only got one extension!
(Skipping over extensions slowly means searching for the next extension
header.)  In our experience, RPFITS contains multiple HDUs of unspecified
length and skipping through them has never been a problem.

Your solutions are complicated, hardware-based workarounds to what is
essentially a simple software problem.  We already have all the hardware
needed *AND* a simple software solution - RPFITS, but the aim is to move
away from that to something more portable.

Bill's solution involves creating multiple bintables in fixed-size chunks
and I agree that this may have other advantages.  However, the last table
in the set has to lie about the true number of rows.  What should a general
(as opposed to special purpose) table reader do on encountering the
null-filled rows?  Is it meant to recognize that this is not real data and
adjust its value of the number of rows to suit?  I didn't find the answers
in NOST.

This raises another question: what should a general bintable reader do if
it encounters an EOF before reading the number of rows specified by NAXIS2?
Hopefully not crash!  So should it adjust its own value of the number of
rows to reflect what it found (if it matters)?  If so, could we instead set
NAXIS2 to some huge value and rely on the EOF?

>NAXIS2=-1 will cause an incorrect calculation of the length of the
>BINTABLE extension.  Any reading application which tries to skip the
>extension is likely to fail in some 'interesting' way, unless it has
>been programmed to recognize this idiosyncrasy.  Of course, a reader
>which applies a strict interpretation of the rules would declare a
>negative NAXIS2 dimensionality to be in violation of the explicit rule
>for NAXISn in Section 5.4.1.1 of the NOST Definition of FITS.

...and it can't be changed??

Cheers, Mark