[fitsbits] ASCII tables, TFORM and recommended practice

[Tom McGlynn]

Phil Hodge wrote:
> Tom McGlynn wrote:
> 
> 
>>1. It implies that columns should be written with a constant absolute
>>precision. ...
>>
>>2. It requires that the data be analyzed to get the appropriate precision
>>before writing any data. ...
>>
>>3.  FITS ASCII tables are particularly susceptible to being edited by
>>simple software tools including text editors. ...
>>
>>The nice thing about the Fxx.0 formats is that they say treat the number
>>exactly as it's written, so there are no surprises. ...
> 
> 
> Those are all good reasons for using binary tables. 


Not really.   Binary tables only allow a single choice (well two I guess)
choices for the relative precision 10^-7 and 10^-16.   If a user has data that's
good to say 1% they are out of luck.

Regardless, for much
of the software I'm writing, I leave it up to the user to select whether
or not they want to use an ASCII table.  When they choose ASCII tables I can't
silently change it to binary.

>...If you really
> need to use ASCII tables, you're stuck with their limitations.  One
> other argument for not using Fxx.0 format is that a table editor
> (i.e. not something like emacs) that reads the data and rewrites it
> would be obligated to rewrite the floating point columns with the
> decimal part rounded off, because that's what the format is.

Well, I guess I don't this as being a limitation of ASCII tables, but one
of avoiding dangerous usage.  The tables I'm writing are legal  (unless I've
missed something), so it's perfectly feasible for data in an ASCII table
to have differing numbers of decimal points in a given column.  The problem
is that the TFORM keyword is being used for 5 different purposes.

The standard indicates that
1. The letter gives the internal type
2. The number before the decimal (for float types)  is used to define the length of the field
3. The value after the decimal is used to define the position of the interpolated
decimal if no decimal is seen in the string.

Some software also uses
4. The full format is used as a default format for display

but that's not really in the standard.

What I'd like to do is reserve TFORM for function 1 and 2.
Strongly deprecate #2 (by using nn.0)
and use something else for #3.

Bill Pence in a message that has gone through Fitsbits but not to the
newsgroup yet doesn't think much of my concern that interpolated
decimals may sneak into FITS files, but does suggest that TDISP should
be used in ASCII tables for function #3.  That would be a great
solution from my perspective if generally agreed upon.

	Tom