[fitsbits] Bintable proposals

[Mark Calabretta]

On Fri 2001/11/02 14:57:07 CDT, William Pence wrote
in a message to: FITSBITS <fitsbits at cv.nrao.edu>

>Does anyone have any comments on these 2 proposals before they are submitted
>to the WGAS FITS committee?

Greetings,

I have used binary tables extensively for single dish data, including use
of the variable length array convention.  In fact I instigated the use of
variable length arrays in SDFITS.  I found their implementation within
cfitsio to be virtually transparent.

However, I have a comment on their use which relates to the requirement
that NAXIS2 store the number of rows in the table.

The fact that NAXIS2 appears in the header and has no default value means
that when writing to a streaming device such as a tape drive the size of
the output binary table must be known before writing can commence.  In
practice, this precludes the use of binary tables for acquisition of data
without using a disk as intermediary since the table size is not generally
known in advance.

The application in question (at Parkes) involves dumping prodigious amounts
of pulsar pulse profile data directly to DLT at close to its maximum I/O
rate.  It was not considered feasible to stage this data to disk.

However, since each table row is a fixed length and the binary table is the
only extension, one solution is simply to write NAXIS2 = -1 and let the FITS
reader find the eof.  Later on, as part of the processing, this invalid
binary table, or the products thereof, could be made legal by rewriting the
true value of NAXIS2.  So given the choice either of setting NAXIS2 = -1 or
avoiding the use of binary tables altogether we opted for the former.

In general, the only requirement is to be able to determine the end of the
table and I can only think of the following impediments:

   1) Extensions following the BINARY table,

   2) Table row length less than 2880 bytes (i.e. whole table rows may fit
      in the block padding),

   3) Use of variable length arrays.

(1) and (2) do not affect us but anyway they do not appear to be basic
problems.  Only (3) is fundamentally limiting.

Since we're using a non-standard construct and don't intend to write
variable length arrays their standardization is not an issue for us and
this note should not be considered an objection to that.  However, if
NAXIS2 = -1 ever did become standard usage it would have to preclude the
use of variable length arrays.

Mark Calabretta
ATNF