[fitsbits] start of Public Comment Period on compressed FITS image and tables

[Lucio Chiappetti]

On Wed, 24 Jun 2015, Mark Taylor wrote:

> I'd like to express my feeling that: it looks pretty complicated.

It is indeed

> One of the attractions of the FITS format (as opposed to [...] is that 
> access to the basic numeric data is very straightforward

Yes, that's even stated in the standard itself "One important feature of 
the FITS format is that its structure, down to the bit level, is 
completely specified in documents..."

I had my own reservation (as a "simple-minded user") about the ZIMAGE and 
ZTABLE conventions (let us call them so for short), but in this respect I 
feel the document is complete.

Which does not mean it will be easier to implement in any language, nor 
that any reader shall implement it.

I am a "simple-minded user" who is NOT managing a big archive, but likes 
to tamper with FITS files using homegrown programs in a language of 
choice. Which usually support only the features I need. As such I will 
*never* embark in writing myself code for ZIMAGE and ZTABLE,

But I know I can rely on utilities like fpack/funpack so I would be able 
to handle a ZIMAGE or ZTABLE using such "external" command (the same way I 
handle most of .gz .bz2 or .zip).

On the other hand I appreciate that persons managing a big data centre may 
be interested in (a "standard") compression. Most likely I would suggest 
to consider ZIMAGE and ZTABLE (even if naively with one-time fpack) even 
to people managing a small archive, knowing the users may always funpack.

> but three weeks is not enough time to attempt an implementation to check 
> it does actually provide all the description required to implement these 
> conventions.

That statement reminds me interoperability testing is part of the criteria 
to be considered by the EC when going to votes. I assumed the fact the 
registered conventions (in general) have been around for quite a while 
could waive for most of the interoperability demonstration.

Nevertheless, while nobody is asking that topcat supports ZIMAGE and 
ZTABLE now, in 3 weeks, 3 months or 3 years :-) it would be helpful if the 
original authors and supporters of the convention may recall the status of 
interoperability (multiple implementations), and also any other author of 
popular s/w (e.g. ds9) may state whether they support it already.


> The text looks like what it presumably is, a post-hoc codification of a 
> series of experiments in compression

This seems a bit unfair or un-kind, but ...

> I am aware there are some committed advocates and strong arguments for 
> use of this convention.

... yes, I think the original authors shall spell out all the arguments in 
support during this discussion.

> is all there in CFITSIO, but that can't be used directly by 
> non-C-friendly languages such as java, javascript, and who knows

is Fortran C-friendly ? or idl ? I would not engage myself in supporting 
ZIMAGE or ZTABLE directly, not even in java ... I guess it won't be very 
efficient. Although if somebody writes a class for a Rice (or PLIO or 
HCOMPRESS) stream (gzip *is* supported) the problem can be nicely 
isolated.

(well javascript is not a serious language really suited to deal with 
binary data, is it ?)

And after all a way out is there (funpack before or fpack after 
processing "normally")

> browser-based FITS image viewers which currently can display any legal 
> FITS image

Having written myself an adhoc viewer in java which can display both a 
local and http FITS image ... well, maybe an adhoc one is different. I did 
the choice which WCS to support and which not. In an adhoc tool one 
chooses which conventions are supported.

I guess in this case it may not be immediate to uncompress on the fly (but 
a retrieve, funpack then display from a local file:// would work)


-- 
------------------------------------------------------------------------
Lucio Chiappetti - INAF/IASF - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.iasf-milano.inaf.it/~lucio/personal.html
------------------------------------------------------------------------
Do not like Firefox >=29 ?  Get Pale Moon !  http://www.palemoon.org