[fitsbits] start of Public Comment Period on CONTINUE Long Kwd convention

[Tom McGlynn (NASA/GSFC Code 660.1)]

Mark Calabretta wrote:
>
>>> The current proposed change to the standard allows *any* string-valued
>>> keyword to be continued, with six exceptions: XTENSION, TFORMn, EXTNAME,
>>> TTYPEn, TDISPn, and TNULL.  Potentially, therefore, FITS readers must
>>> always use fits_read_key_longstr(), or its equivalent, to read a
>>> string-valued keyword (except for the six), and will be responsible for
>>> freeing the memory allocated for each and every one.  The same goes for
>>> keycomments also if they are allowed to grow in size.  There will also
>>> be side-effects arising from the need to change the way that string
>>> keyvalues are stored, e.g. as struct members.
>> I'm a little confused by this.  When long strings are supported I would
>> assume a program that supports them (including using CFITSIO) reads the
>> string value using it's standard mechanism and if it sees a trailing '&'
>> some special software kicks in.  For the vast majority of string values,
>> no special handling need be done.   Right now CFITSIO supports long
>> strings for virtually any string keyword. Nothing is changing here.
>> Clearly libraries which don't support long strings will need some
>> update, but for the libraries which do the only different may be that
>> the support is always enabled rather than being triggered by the
>> LONGSTRN keyword or some user flag.
> The FITS standard defines the data type for all recognised keywords,
> and this allows an application programmer to decide in advance how to
> read a keyvalue.  So, for example, CRVALia is floating point and
> WCSNAMEa is a string limited to 68 characters in length.
>
> Using cfitsio, to read CRVALia the app programmer currently uses
> fits_read_key_dbl() (or equivalently fits_read_key(..., TDOUBLE, ...),
> or possibly the float equivalents with the potential loss of some
> precision, and provides an appropriate variable to hold the value.
>
> Currently, the appropriate function to read WCSNAMEa is
> fits_read_key_str() (or equivalently, fits_read_key(..., TSTRING, ...)),
> and the programmer provides a char[] of at least 68+1 characters to hold
> it.  However, if WCSNAMEa were allowed to grow indefinitely long, then
> this would no longer be adequate, essentially the resulting "loss of
> precision" here equates to losing the nether part of the string, and
> existing code would potentially be broken.  To repair it, or when
> writing new code, because the programmer has no way of knowing in
> advance whether WCSNAMEa will be continued or not, he must assume the
> worst and always use fits_read_key_longstr(), which works in quite a
> different way to fits_read_key_str().  In fact, the latter function
> would no longer be appropriate for reading anything other than XTENSION,
> TFORMn, EXTNAME, TTYPEn, TDISPn, and TNULL.

It seems that your beef here is with FITSIO and its implementation of 
long strings.  There is nothing in the standard which precludes long 
string values and regular values from being retrieved using the same 
method -- that's the case in at least some other FITS reading 
libraries.  And even with FITSIO I expect that it would be easy to write 
a routine with code with something like the following language.  (Please 
forgive the bastardized C syntax).

     char *getRegularOrLongString(header, key) {
        char *val;
        fits_read_key_string(header,key, val,..)
        if (lastCharPointedToByValIsAmpersand) {
            *val = fits_read_key_longstr(....)
       }
        return val;
    }

I gather that there are garbage collection issues potentially but 
presumably that could be addressed by a more sophisticated version of 
the routine above.   This would mean that code that uses CFITSIO would 
need to be updated eventually.  I don't know that it would be urgent 
since no one currently has occasion to use these long values.


>
> In this respect, FITS strings and continued strings are different data
> types.  The CONTINUE proposal, as it now reads, effectively amounts to
> changing the data type of all string-valued keywords recognised by the
> standard, apart from those six.

My inclination would be to minimize the 'specialness' of long strings 
and allow them almost everywhere -- except for keywords that are 
required to be in fixed format.  But just as you may be influenced by 
the CFITSIO implementation, my instincts may be affected by the 
uniformity of the string handling in the Java FITS libraries where 
(other than currently having to enable the convention for writing) long 
header strings are handled largely transparently to the user.

     Regards,
     Tom McGlynn
> To be clear, I'm not against CONTINUE, just the current wording of the
> proposed addition to the standard.  Also, I am curious as to how the
> equivalent of fits_read_key_longstr() would be implemented in Fortran.
>
> Regards,
> Mark Calabretta