[fitsbits] Proposed Changes to the FITS Standard

[Rob Seaman]

Bill said:

> The "once FITS always FITS" philosophy captures the spirit of FITS,  
> but
> in practice each new version of the FITS Standard has imposed new
> requirements that in principle could invalidate existing FITS files.
> For example, version 2.0 of the FITS Standard introduced a new
> requirement that the value and comment fields in a keyword MUST be
> separated by a slash character.

It would be interesting to review past such instances.  I don't  
personally recall changes of this mandatory nature.  The example  
regarding comments is pretty tame since any reasonable implementation  
would already be ignoring the comments.  Do you have another example  
to quote?

> Of course, if the FITS community thinks a new requirement would cause
> too much dislocation to existing data or software, then an alternative
> would be to just "strongly recommend" instead of "require" the new
> feature.

Indeed.  I think that would be best in all three instances quoted below.

> It's also possible to specify that a new requirement will come
> into effect at some point in the future to allow time for software
> systems to adapt, as was done with the Y2000 change to the
> DATE keyword format.

Obviously the scheduling for the Y2K changes was forced, but I agree  
with Steve that any such new requirements should be announced well in  
advance of taking effect.

I think, however, that there is a misapprehension about the DATE/DATE- 
OBS changes.  The new ISO date format was very carefully designed to  
only be required for post-Y2K data (there was also some overlap  
period as I recall).  The old format remained - and remains - valid  
to describe 20th century data.  In fact, the old dd/mm/yy format was  
clarified to explicitly denote such dates.  No after-the-fact  
requirements were leveraged onto archival data.  This is very  
different than attempting to place new absolute requirements that  
would invalidate old data sets.  I say "attempting to place", because  
there is no mechanism for enforcement.

>   There are only 3 proposed new absolute requirements in this list:
>
>   1. Keywords that have a value shall not be repeated in a header.

I have many examples (hundreds of thousands?) of files in which  
keywords are repeated.  Rather than the wording in the current  
proposal, I would replace the attempt at a requirement with a strong  
recommendation and a clarification that the final copy of any such  
repeated keyword should take precedence.

>   2. PCOUNT and GCOUNT must immediately follow the last NAXISn
>      keyword in all conforming extensions (as is already required
>      in IMAGE, TABLE, and BINTABLE extensions).

I guess I'd like to know if there are any such extensions.  If not,  
this is relatively safe.  If so, make it a strong recommendation for  
an explicit list of grandfathered extension types and an absolute  
requirement for any newly defined extensions.

>   3. Embedded space characters are now forbidden within numeric
>      values in an ASCII Table (e.g.  "1 23 4.5"  is no longer
>      allowed to represent the decimal value 1234.5)

Again - are there any examples of such usage in the field?

I think the general principle, however, should reflect the "letter of  
the law", not "spirit of the law".

I should end here by repeating my earlier appreciation of the  
excellent effort that has gone into the revision.  If this careful  
revision has not uncovered any other critical new requirements that  
must be applied ex post facto, one can opine that there are no  
lurking dragons that need to be fought.  That being the case, it  
seems to me that the responsibility lies rather to preserve the great  
investment in archival data products rather than to attempt to  
legislate these new requirements on the back of our current holdings  
and current software investment.

And should new dragons appear that the community deems must be slain,  
it does indeed appear to this observer that an explicit version  
keyword (whether a comment or not) should be simultaneously required  
to trigger new conformance restrictions.  The loose wording about pre- 
existing data is unenforceable since there is no requirement (whether  
or not there ought to be) for a DATE keyword to separate old from  
new.  Perhaps the new version tag could itself supply a date - in  
that case, I'd recommend that any revisions of the standard should  
contain explicit references to the date(s) that apply for different  
feature(s).

Rob