[fitsbits] reopening of Public Comment Period on the Green Bank convention

[Tom McGlynn (NASA/GSFC Code 660.1)]

I am strongly opposed to adopting this convention within the FITS 
standard although I think it is perfectly appropriate for projects to 
use as they wish.

The problem I see is that the convention is too ill-defined and strongly 
and undesirably couples the FITS header and content. From the 
perspective of software that is to consume FITS files the implication is 
that if a user is reading a binary table and the user asks for a column 
which is not present in the table but which matches any header field, 
then the software should return data as if the column is found using the 
value in the header.   This is a immense change to software and means 
that the reader can't simply use the header to determine the metadata 
for the columns, it may turn out that there are columns hidden there.  
And there are lots of arbitrary choices so we have to carry the header 
along forever.  Are integer values 1, 2, 4 or 8 bytes?  Are reals 4 or 
8?  What's to be done if a there is a keyword with an explicitly null 
value?  What limits do we impose: Can the user ask for BITPIX as a column?

If this convention were restricted to some specific list of keywords it 
would be less problematic, but as part of the standard I'd see it as a 
kludge.

I hadn't understood the Greenbank convention as going the other way: 
replacing a header keyword with a column in the table, but I also think 
this is misguided.  For keywords which are not defined in the FITS 
standards, the user is already free to define such a column.  For fields 
normally used in some scalar context and bound to a column for use in a 
table like the WCS keyword in the example: these should be explicitly 
addressed by the relevant sections of the document (I don't recall if 
they are).   So the WCS discussion should explicitly talk about the use 
of WCSs in tables.  Of course the proposed approach really doesn't solve 
the problem since in principle a single row could have many images each 
of which needs different WCS values but this approach only allows a 
single WCS for a given row. The approach here is a band-aid on the lack 
of true structures in FITS.

As a convention used by different groups which each have different 
choices as to what columns may be compressed to keywords I think it is 
great, but I don't believe that all useful conventions need to be or 
should be incorporated into the standard.

     Regards,
     Tom McGlynn

Lucio Chiappetti wrote:
> ANNOUNCEMENT:  START OF FORMAL PUBLIC COMMENT PERIOD
>
> This is to announce the official reopening of a formal Public Comment 
> Period to recognize and mention the Green Bank convention in the FITS 
> Standard document,
>
> A previous PCP was held in June-July (started on 19 Jun 2015).
>
> We are now re-opening the Public Comment Period to report the 
> situation before passing to a vote to modify the standard document.
>
> The Green Bank convention allows to expand a kewyord in a table column 
> (or collapse a column with identical values in a keyword).
>
> During the previous PCP, different opinions were expressed towards the 
> original proposal, which aimed to *incorporate* in the standard the 
> Green Bank convention, originally mentioned in the WCS papers, in a 
> section or cross reference in chapter 7,
>
> Since the main idea was to simply *document* the existence of the 
> historically significant Green Bank convention in the FITS Standard, 
> we are now just falling back to the exact wording that was used in the 
> WCS paper, which implies NO CHANGE to the standard.
>
> The proposed text consists of the paragraph in green in section 8,2 as 
> presented in the draft at this link 
> http://sax.iasf-milano.inaf.it/~lucio/FITS/Conventions/greenbank-rev2.pdf
>
> The text contains in the second page also an excerpt of Appendix H.3, 
> which now makes clear the Green Bank convention remains a registered 
> convention, which is recognized by the standard.
>
> Since the IAU FWG has already approved the WCS papers as part of the 
> FITS Standard in 2003, and we are just inserting in the main document 
> a bit of text from a WCS paper (section 3.6 of WCS paper I), this is 
> just an EDITORIAL change, and as such will require just a plain 
> procedural vote.
>
> Please review the text carefully and post any comments, criticisms, or 
> suggestions on the FITSBITS mailing list.
>
> Supporting material for the PREVIOUS public comment period 
> announcement can be retrieved from the FITSBITS archives starting from 
> https://listmgr.nrao.edu/pipermail/fitsbits/2015-June/002620.html
>
> ==================================================================
>
> The reopened Public Comment Period starts today 25 May 2016 and will 
> last until 5 June 2016, unless extensions are requested and motivated.
>