[fitsbits] further reopening of Public Comment Period on the CONTINUE convention

[Mark Calabretta]

On Tue, 19 Apr 2016 20:21:13 -0400
William Pence <William.Pence at nasa.gov> wrote:

> For one thing, if the CONTINUE 
> convention is used to represent a string value that is less then 68 
> characters long (and hence could have been represented on a single 
> header keyword), then this is simply a different way to represent (or 
> serialize) the exact same character string.

Not quite.  If a string keyvalue is allowed to be continued, then it
forces software always to treat it as though it may be of indefinite
length, whether or not it actually is, and regardless of whether it
is needlessly split over multiple keyrecords.

> Similarly, it is hard to see why a CONTINUE'd string value that is 69 
> characters long should be considered a different data type from a string 
> value that is 68 characters long (whether it is represented as a single 
> keyword or uses the CONTINUE'd convention).

When reading a string keyvalue in C, the result is guaranteed to fit in
a char[69] (with null-termination) and can be read via

  char my_string[69];

  get_string_keyvalue(..., my_string, ...);

Something similar applies in Fortran 77 (AIPS, Miriad).

However, since the length of a CONTINUE'd string is unknown, memory must
be allocated for it, in this example I assume by the library routine that
reads the keyvalue, and freed by the application program:

  char *my_long_string;

  get_longstring_keyvalue(..., &my_long_string, ...);

  free(my_long_string);

There is no equivalent for this in Fortran 77.

So, while both my_string and my_long_string are basically pointers to
char, the interpretation and usage differs.  The difference becomes more
obvious when they are embedded in structs.

> I think the most that can be said is that the string length *might* 
> present some language-dependent software implementation issues,

*might* -> *does*

>but this 
> is not a fundamental property of the FITS keyword itself.

It fundamentally affects the way software has to deal with them.
Whether that translates to being a fundamental property of the FITS
keyword itself I will leave to the philosophers.

> For example, 
> the string length is a non-issue for programs written in java because 
> there is no need to allocate or free the memory needed to store the 
> string value.

In a perfect world, everything would be written in <insert your
favourite language here>.

> Memory management for the string values can be more of an issue with C 
> and Fortran programs, but it depends greatly on exactly how the program 
> is written.  For example, if the program calls the so-called 
> 'long-string' keyword reading routine in CFITSIO, then CFITSIO itself 
> takes care of most of the memory management issues.  The next release of 
> CFITSIO, next week, will also introduce a new string keyword reading 
> routine which lets the reading program decide the maximum length of 
> string that it is prepared to read, thus eliminating the arbitrary 
> 68-character demarkation point.

Most of the reserved keywords in the standard would never need to be
continued.  Allowing it would introduce an extra, unwarranted, level of
complexity for FITS readers.

> Finally, perhaps the most important thing to keep in mind is that when 
> the CONTINUE convention was invented 20-some years ago, the HEASARC 
> *never* intended to use it with any of the string-valued keywords that 
> were already defined in the FITS standard.  It was *only* intended for 
> use for new mission-dependent keywords that were created by the HEASARC. 
>   This is why I suggested adding that sentence that prohibits using 
> CONTINUE with any of the existing keywords unless explicitly stated 
> otherwise.  If and when new reserved string-valued keywords are added to 
> the standard, a decision can be made at that time as to whether that 
> keyword may be CONTINUE'd or not.

So it seems we now agree on creating two classes of string-valued
keywords.  I would simply like to assign the labels "string" and "long
string" to them.

Regards,
Mark Calabretta