[fitsbits] Bintable proposals

[Mark Calabretta]

On Wed 2001/11/14 07:09:31 CDT, Bill Cotton wrote
in a message to: Mark Calabretta <Mark.Calabretta at atnf.CSIRO.AU>
and copied to: William Pence <pence at tetra.gsfc.nasa.gov>, FITSBITS <fitsbits at nr
     ao.edu>

>   What you address is a fundamental FITS limitation rather than just
>a binary tables one.  Each header is required to say what the length
>of its data unit is.  This does present problems for data acquisition.

Hi Bill,

Am I correct in saying that this arises from the fact that Modcomps could
not do file expansion, i.e. the correct file size needed to be allocated
before writing could commence?  [For those who've never heard of them,
Modcomps were/are an ancient type of computer mainly used for realtime
applications.]  

As I understand it the history goes something like this: the genesis of
FITS is linked to AIPS; AIPS was intended to run on Modcomps; thus early
versions of AIPS did not have file expansion; therefore FITS didn't assume
it.  This limitation has never been relaxed.

Does it seem reasonable to preserve such a fundamental limitation on FITS
given that the original reason for its existence no longer applies?

With regard to random groups data, the ATNF got around it from the start
(c1985) by inventing a FITS-like format called RPFITS (Ray's Pseudo-FITS,
as Eric calls it) which doesn't require the data size to be specified in
the header.  This aspect of RPFITS has never caused any problems for its
readers, including the venerable AIPS task ATLOD (c1985).  However, RPFITS
is not widely recognized and has sufficiently many other idiosyncrasies
(i.e. drawbacks) that we would like to move away from it to something more
portable.  Hence we decided it would be better to bend the rules on binary
tables hoping that the rest of the world might catch up some day.  Part of
the rationale is that a BINTABLE reader which can't grok NAXIS2 = -1 would
skip the file and be no worse off contra a totally unrecognized format.

Cheers, Mark