[fitsbits] Abuse of EXTEND keyword

[Tim Pearson]

On Aug 23, 2007, at 12:18 PM, William Pence wrote:

> Tim Pearson wrote:
>> Perhaps I am misunderstanding or misremembering something here, but I
>> think EXTEND=T does actually have a purpose, and to suggest that both
>> EXTEND=T and EXTEND=F are merely advisory is crazy. We should not add
>> the words "nor does the absence of this keyword necessarily imply
>> that the file does not contain extensions" to the FITS standard.
>
> Can you explain why this should not be done?  Also, the EXTEND = T
> keyword has always been advisory, so it seems consistent to declare  
> that
> EXTEND = F should also be advisory.

EXTEND=T when introduced was not advisory: it was an explicit  
indication that all the "special records" (if any were present) after  
the primary HDU contained only standard extensions. It would not have  
been introduced if it was not necessary to distinguish such well- 
formed files from those that contain arbitrary (unspecified) data in  
the special records.

Sometime since the Grosbøl paper, it appears that two things crept  
into the standard:

1. Special records that are not standard extensions can appear after  
the last standard extension, even if EXTEND=T. I don't like this: it  
is unnecessary and ugly and makes it harder to detect corrupted  
files.  Perhaps it is too late to undo it (although it is now  
"deprecated").

2. Special records that are not standard extensions are not allowed  
to start with 'XTENSION'. This is stated in section 3.5 of the  
standard. It is this change that allows a reader to ignore the EXTEND  
keyword and just look for standard extensions in the data stream.  
This change would not have been necessary if change 1 had not been  
made. I suppose that the chance that this change invalidated an  
existing FITS file is negligible. When I wrote the above, I had not  
noticed or had forgotten this statement in Section 3.5.

With these two changes, I agree that EXTEND is no longer needed. I  
see no point in attaching "advisory" meaning to its presence,  
absence, or value (what should a program do based on this advisory  
information?); just say it conveys no information and should be ignored.

- Tim Pearson