[fitsbits] FITS nit: variable length arrays and TDIM.

[William Pence]

tam wrote:
> 
> There's a bit of an inconsistency/incompleteness in the NOST appendices
> for conventions for binary tables.
> 
> In appendix B.1 on varying length columns we have:
> 
>     Conventions such  as TDIMn (see Appendix B.2) apply equally to
>     both variable length and static arrays.
> 
> and in appendix B.2 we have:
> 
>     The size implied by the TDIMn keyword will equal the element
>     count specified in the TFORMn keyword.
> 
> Now I'm a little confused as to what it would mean to have a variable length
> array with a TDIM keyword.  What is the element count specified in TFORM for that?
> Is it the 0 or 1 at the beginning of the TFORM or the potentially absent maximum size
> specified in the at the end?

I agree this is confusing and section B.2 should probably be clarified.

> What happens when the actual length of the column varies from
> the product of the TDIM factors?  Are we supposed to extend the array
> with 0's to fill up the array?  Do we mean that the first or last dimension
> is variable [but then what if the actual size is relatively prime to the
> TDIM factors]?  I can imagine a number of reasonable solutions, but for
> the moment I don't believe one can use TDIM with varying length columns
> since there doesn't seem to be a definition of what that means.

The sentence in B.2 says that the size implied by TDIMn *WILL EQUAL* the size
of the array;  if not, then this is an error that violates the proposed use of
this keyword.  Software should probably ignore the TDIMn keyword if this error
occurs.

> Does anyone actually have any example of where TDIM was used intelligently with
> a variable length array?

There are some trivial cases where the TDIMn keyword could apply: (1) The
table has only one row, or (2)
the table has more than one row, but the variable length array is the same
length in every row.  In these cases, however, there is no advantage to using
a variable length array instead of a simpler fixed length array.

One realistic example I can think of is where the variable length array is the
same size in all the rows, except that the array is not present (has zero
length) for some of the rows.  In this case, you would want to use variable
length arrays to save space in the rows where the array is not present, but
you would use the TDIMn keyword to specify the (fixed) size of the array in
the rows where it is present.

> Would it be better to have B.1 say that variable length arrays are always
> one dimensional?

No.  This would unnecessarily restrict the use of variable length arrays.

>  Or do we need to add something to B.2 to specify what TDIM means
> in the context of a variable length element?

This is what I would recommend, perhaps to simply say:

"The size implied by the TDIMn keyword will equal the total length of the
array."

Is it really necessary to specify here exactly how one determines the length
of the array?

Fortunately, Appendix B is "not part of the NOST FITS Standard but is included
for informational purposes only" so this issue should not impact the formal
approval of the Standard that is now in progress.

-Bill Pence
____________________________________________________________________
Dr. William Pence                          pence at tetra.gsfc.nasa.gov
NASA/GSFC Code 662         HEASARC         +1-301-286-4599 (voice)     
Greenbelt MD 20771                         +1-301-286-1684 (fax)