[fitsbits] TDIMn

[Preben Grosbol]

On Wednesday 25 August 2004 19:04, William Pence wrote:
> This motion proposes to incorporate Appendix B.1 and B.2 into the official
> FITS Standard  (the appendices are currently unofficial).  Both appendices
> apply to FITS binary tables: B.1 describes the Variable Length Array
> facility, and B.2 describes the TDIMn keyword. (See
> http://archive.stsci.edu/fits/fits_standard/node74.html).  There is also a
> third appendix, B.3, dealing with substring arrays, but only a portion of
> this proposed convention has ever been used or implemented (to my
> knowledge), so only part of this appendix, if any, should be considered for
> incorporation into the FITS Standard.
>
> If anyone has any issues or concerns about making these appendices part of
> the official FITS format Standard, now would be a good time to raise them
> here on this newsgroup.  The FITS committees will likely be considering
> this issue soon, later this year.

I see no problems with the B.2 (TDIMn) proposal as it only deals with the
interpretation of a one dimensional array.  Certainly it would be good to
have the appendix B.2 included officially into the standard soon.

With respect to B.1 (variable length arrays), I have some reservations.
This proposal requires a full implementation of the HEAP and the P column
format.  Technically, the B.1 proposal is good and I would endorse it as such.
The issue is that (as least to my taste) FITS files which conform to the
standard should be readable by most readers.  The success of FITS is
to some extend based on this, namely that one having a standard FITS
file does not need to worry - it is, by default, readable by almost all
systems.

If B.1 is included into the standard I am concerned that a significant number
of systems will be unable to read and access such data.  It is not just a
matter of if HEAPs can be accessed by software libraries but more if full
data processing systems can access such data as if they had been in a
'normal' table column.  Beside leaving B.1 as it is, there may the following
options to include it into the standard:
  1) Adding an clear, explicit preample which states that users cannot assume
      the HEAP concept to be implemented by all readers and therefore should
      be cautious in using it.
 2) Verify that most readers actually already implements the HEAP which
      would satisfy my concern.

On B.3, it would be good to know who is actually using it?  One should
only include, into the standard, features which are used by a broad community.

Preben Grosbol