[fitsbits] FITS nit: variable length arrays and TDIM.

[tam]

I guess I can see three reasonable possibilities:

- The minimal impact to the standard would change B.2 to say "The size implied by
the TDIM keyword will equal the length of the array."  and require that all of
the rows have that length.  After some reflection Bill and I did realize that even
this could potentially be useful when users want to reuse the same array in multiple
rows, i.e., when the varying length column is used to alias elements.

- A slightly larger change would be to do the above, and also allow 0 length arrays.
I.e., a given row can have 0 length or what is specified in TDIM.  This probably needs
some wording added in B.1. as well as the above.  It's what Bill is suggesting.

- A still more generic change would be to change B.2 to say "The size implied by the
TDIM keyword will divide the length of the array evenly".  [Most likely we'd
want to write this as a special case for varying length columns, but it doesn't break anything
to say it generally, i.e. the statement is true for all current valid FITS files.]  
E.g., if I have a something which returns little 2x2 snapshots but
may return a variable number of them I'd specify the TDIMnn='(2,2)' and then
the length of the column should be some multiple of 4.  This handles the case of 0
length also.

Personally I'd be happy to go along with any of these -- I don't think any of them
are hard to implement.  

	Tom McGlynn

William Pence wrote:
> 
> tam wrote:
> >
> > There's a bit of an inconsistency/incompleteness in the NOST appendices
> > for conventions for binary tables.
> >
> > In appendix B.1 on varying length columns we have:
> >
> >     Conventions such  as TDIMn (see Appendix B.2) apply equally to
> >     both variable length and static arrays.
> >
> > and in appendix B.2 we have:
> >
> >     The size implied by the TDIMn keyword will equal the element
> >     count specified in the TFORMn keyword.
> >
> > Now I'm a little confused as to what it would mean to have a variable length
> > array with a TDIM keyword.  What is the element count specified in TFORM for that?
> > Is it the 0 or 1 at the beginning of the TFORM or the potentially absent maximum size
> > specified in the at the end?
> 
> I agree this is confusing and section B.2 should probably be clarified.
> 
> > What happens when the actual length of the column varies from
> > the product of the TDIM factors?  Are we supposed to extend the array
> > with 0's to fill up the array?  Do we mean that the first or last dimension
> > is variable [but then what if the actual size is relatively prime to the
> > TDIM factors]?  I can imagine a number of reasonable solutions, but for
> > the moment I don't believe one can use TDIM with varying length columns
> > since there doesn't seem to be a definition of what that means.
> 
> The sentence in B.2 says that the size implied by TDIMn *WILL EQUAL* the size
> of the array;  if not, then this is an error that violates the proposed use of
> this keyword.  Software should probably ignore the TDIMn keyword if this error
> occurs.
> 
> > Does anyone actually have any example of where TDIM was used intelligently with
> > a variable length array?
> 
> There are some trivial cases where the TDIMn keyword could apply: (1) The
> table has only one row, or (2)
> the table has more than one row, but the variable length array is the same
> length in every row.  In these cases, however, there is no advantage to using
> a variable length array instead of a simpler fixed length array.
> 
> One realistic example I can think of is where the variable length array is the
> same size in all the rows, except that the array is not present (has zero
> length) for some of the rows.  In this case, you would want to use variable
> length arrays to save space in the rows where the array is not present, but
> you would use the TDIMn keyword to specify the (fixed) size of the array in
> the rows where it is present.
> 
> > Would it be better to have B.1 say that variable length arrays are always
> > one dimensional?
> 
> No.  This would unnecessarily restrict the use of variable length arrays.
> 
> >  Or do we need to add something to B.2 to specify what TDIM means
> > in the context of a variable length element?
> 
> This is what I would recommend, perhaps to simply say:
> 
> "The size implied by the TDIMn keyword will equal the total length of the
> array."
> 
> Is it really necessary to specify here exactly how one determines the length
> of the array?
> 
> Fortunately, Appendix B is "not part of the NOST FITS Standard but is included
> for informational purposes only" so this issue should not impact the formal
> approval of the Standard that is now in progress.
> 
> -Bill Pence
> ____________________________________________________________________
> Dr. William Pence                          pence at tetra.gsfc.nasa.gov
> NASA/GSFC Code 662         HEASARC         +1-301-286-4599 (voice)
> Greenbelt MD 20771                         +1-301-286-1684 (fax)