[fitsbits] start of Public Comment Period on compressed FITS image and tables

[Douglas Tody]

The point is just that images can be arbitrarily large, especially once we
start dealing with multidimensional / cube data, which is increasingly
common.  Storing the data for such large images all strung together in
the heap area of a binary table is not appealing.  If we store large
images in separate extensions (or separate files which is much the same
thing) then the storage is localized, and it is straightforward to
employ advanced techniques such as compression.

As noted ealier I don't have any problem with storing modest sized
spectra or images as a simple array within a table record.  A typical
use case might be a table of objects where one stores an image cutout
for each object, and maybe a preview JPEG as well, along with the more
conventional object metadata.  In such a case an image of several
hundred pixels square would not be a problem; even your example of 1K
square is not out of the question although that starts to get large.
But bintable was never intended to be used as a general image storage
format, and does not scale well to very large images.  It should be left
up to the user to decide which format is best for their data and
software.    - Doug


On Fri, 24 Jul 2015, THIERRY FORVEILLE wrote:

> "Douglas Tody" writes:
>>  BINTABLE has its uses
>> as well as a container, e.g., for generic conventional tables certainly,
>> but also for tables that store spectra or smaller images like cutouts
>> wherein the data segment can be represented as a array , but it is not
>> ideal for large datasets containing arbitrary types of objects, each of
>> which can be very large.
> Could you elaborate on what problems you see in BINTABLE being used as a
> container for individually large objects? I know of at least one case of
> its being used for things that were fairly large at the time (~1kx1k images
> in the early 90s), but I may be missing some scaling issues that mean trouble
> at some larger size...
>
> Thierry
>