[fitsbits] ASCII tables, TFORM and recommended practice

[Tom McGlynn]

Phil Hodge wrote:
> Tom McGlynn wrote:
> 
> 
>>My question is basically what do others think on this issue?
> 
> 
> Use an explicit decimal point in the data, and give the actual print
> format as the TFORMn value, e.g. F7.3 in your example.  In my opinion,
> the situation is clearest when the two are consistent.
> 
> Phil
> 

I'd agree that it's nice when everything agrees, but I'm not
sure that it's always possible or desirable.  Here's a little
more detail about why I don't like to use formats with explicit
decimals.


1. It implies that columns should be written with a constant absolute
precision.  In many cases a user may want a constant relative precision
or have other reasons to use different number of decimal places for
different rows.   Generic writers will probably want to use a constant
relative precision (with 7-8 decimal places for BITPIX=-32 and 15-16 for
BITPIX=-64) unless the user specifies some specific value.  [One could
require exponential notation but that's inefficient and ugly.]

2. It requires that the data be analyzed to get the appropriate precision before
writing any data.  This precludes any streaming processing of the FITS file.
[If the number of rows is not known in advance, then full streaming may not be
possible, but the rows can be processed in a streaming fashion, and the
only adjustment needed when the table data has been read will be to the NAXIS1
value.  If the number of rows is known in advance, then full streaming is possible.]


3.  FITS ASCII tables are particularly susceptible to being edited by
simple software tools including text editors.  If a format specifying the number
of decimals is in the header, and the user happens to enter a integer value
in the data, then bad things can happen if the user isn't very careful.  I've
manually modified more than a few FITS files in my time, and this seems like a way
to get rare but very dangerous and hard to track errors in datasets.


The nice thing about the Fxx.0 formats is that they say treat the number
exactly as it's written, so there are no surprises.  If we were writing the
FITS standards today, I do not believe that the decimal interpolation feature
would be included. I've tried to write FITS files that preclude it from
being an issue.  I would like to see the TDISP keyword  as standard
keyword for the ASCII tables so that I could inform display software
in a standardized way.

	Tom