[fitsbits] Five FITS Proposals

[Erik Bray]

On 10/25/2013 03:56 AM, Peter Weilbacher wrote:
> On Wed, 23 Oct 2013, William Pence wrote:
>> 4.  Introduce a new FITS version keyword
>
> This was one of the Thomas et al. suggestions at the BoF, but I didn't
> really get the point. How does it help older FITS readers if a FITSVERS
> keyword is there which they don't understand?

To me, this is the most important of the proposed changes, if anything in FITS 
is changed at all.  This should be a prerequisite to any future changes.

I was at the BoF, and was a little disappointed when someone suggested that 
"FITS has never had any version compatibility issues" (paraphrasing, but that 
was the spirit of the comment).  This is untrue--there are even features in the 
FITS Standard (the Q column type in BINTABLE, for example) that older FITS 
readers would not know how to interpret).

Indeed, a FITSVERS keyword would not be helpful for older FITS readers, but it 
is very helpful to new (JavaScript-based, for example) and actively maintained 
(CFITSIO, PyFITS, etc.) FITS readers.  While I share some of the skepticism of 
others over some of Bill's proposals I'm not strongly against them either so 
long as there is a clear, simple, effectively O(1) means of determining what 
FITS features need to be supported in order to read a given file.

I would go one further in fact:  While a FITSVERS keyword is useful for 
indicating what version of the FITS Standard is used by a file, it would also be 
very helpful to have standard keyword(s) for specifying exactly what conventions 
(registered or not) that file is using.  Perhaps this would only be required in 
the PRIMARY header, and propagate to all other headers in the file.

For example, if a header uses CONTINUE cards there *must* be an indicator in the 
header like

FITSVERS= '     3.1'
FITSCV1 = 'CONTINUE 2007-07-11'

in order for the CONTINUE keywords to be recognized at all.  Otherwise they 
would be treated as an unknown commentary-like keyword.  Similarly for HIERARCH, 
or any other existing or proposed new conventions.

The reason for this is that current FITS readers have to do a lot of extra work 
when parsing a header to determine what conventions are being used.  Some of 
them, like the CONTINUE card, are mostly unambiguous and not a terrible 
hardship.  But others involve a lot of ambiguity and guessing.  My go to example 
of this is the "record-valued keyword" convention.  Even HIERARCH, believe it or 
not, involves more ambiguity that one might naively think.

While having convention indicators would obviously *not* help with older FITS 
files, if combined with the FITSVERS keyword they can make reading of new FITS 
files easier.  As the implementer of PyFITS, I can just check FITSVERS--if it's 
missing or an older version I turn on preliminary support for all supported FITS 
conventions by default.  Otherwise, is FITSVERS is such that convention 
indicators are supported I can turn on support for specific features in a 
piecemeal fashion, likely resulting in much faster and simpler parsing of 
headers depending on which conventions are enabled.

This would also still work fine with older FITS readers, as they would simply 
ignore the FITSVERS keyword and friends, and would continue to read the header 
as best they can.

Best,

Erik