[fitsbits] reopening of Public Comment Period on the compression conventions

[Lucio Chiappetti]

On Mon, 11 Jan 2016, Tom McGlynn (NASA/GSFC Code 660.1) wrote:

> 1. "Tiled images"  defines a new representation of images as tiles
>    within a binary table. These tiles represented in this standard are
>    not compressed.  They are just fields in a binary table.

Personally I do not see much reason in having these tiled images (well, I 
do recall "Exosat small maps", which were somehow optimized for random 
access, but that was in 1980s), but apparently they are implicity already 
allowed by the secret option Bill kept in his pocket "'NOCOMPRESS' ... 
already supported by CFITSIO and fpack"

> 2. "Table compression" defines a general standard for compression of
>    binary tables.  In this context the tiled images are simply one kind
>    of binary table.

> A file that uses both of these conventions would look like our current
> compressed tile images.

I am not deep enough in the inner working of compression. Is it true that

  - ZIMAGE compressing (storing) an image in NOCOMPRESS tiles
  - then ZTABLE compressing the resulting bintable

  gives the same result as

  - ZIMAGE compressing the image with one particular algorithm ?

If the operations are not associative and the resulting files will be 
different, this should be against the separation. given that there are 
plenty of image viewers supporting current ZIMAGE-compressed images.

> Supporting image tiling seems important, but it's less clear that there 
> much push from the community for supporting non-image compressed binary 
> tables and I would prefer to leave that out until have a few 
> instantiated use-cases.

Essentially I see a benefit of elegance in having both conventions 
described in the same chapter and voted at the same time.

And since FITS has a fame of being slow in its approval process, I would 
avoid things (like we did in the past for BINTABLE) as having some parts 
listed in an appendix as "not part of the standard" and being voted as 
such a few years later. Let's be daring, and pass all in one go.

On Mon, 11 Jan 2016, William Pence wrote:
> would suggest a simpler solution in this case:  Just pipe the compressed 
> FITS file to an external uncompression program (funpack in this case)

This I surely second.

Personally I would never write a line of code about compressing and 
uncompressing, but since I recently received (actually I had to ftp) some 
tens of bulky catalogues which I'll have to store and redistribute, I was 
contemplating to fpack them before storing !

It takes just a couple of minutes to build fpack/funpack after all !

On Mon, 11 Jan 2016, William Pence wrote:

> I support the suggestion by Mark Taylor and Tom McGlynn to decouple the 
> Image compression and Table compression proposals ... the proposals can 
> be voted on separately.

This is a possibility. I note however that some of the IAUFWG voters are 
requesting to move on and speed up things. So I suggest we hold separate 
votes but simultaneously.

It looks to me that Images correspond to 10.1, 10.2 and 10.4, while Tables 
correspond just to 10.3 with a cross-reference to a subset of the 
algorithms decribed in 10.4

Apart from the numbering (if 10.3 is not approved, current 10.4 has to be 
renumbered, OR the order of sections has to be changed beforehand) the 
relevant text is enough segregated.

I'd like to hear the opinion of Dick Shaw which spent a lot of time on 
writing chapter 10 combining the two conventions.

Lucio

-- 
------------------------------------------------------------------------
Lucio Chiappetti - INAF/IASF - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.iasf-milano.inaf.it/~lucio/personal.html
------------------------------------------------------------------------
Do not like Firefox >=29 ?  Get Pale Moon !  http://www.palemoon.org