[fitsbits] Rice compression from the command line

[Mark Calabretta]

On Wed 2006/07/19 07:44:15 MST, Rob Seaman wrote
in a message to: fitsbits at nrao.edu

>1) market penetration - gzip is a clear leader here

I agree that 7zip would be out of the comfort zone of most unix users,
e.g. no Debian package or RPM that I could locate.  However,
installation from source code is simple (configure/make).  It is
already widely used on Windows and Macs and I expect unix will catch
on soon enough.  In any case, for FITS it would be a matter of
integrating the compressor/decompressor (= coder/decoder = codec) into
the FITS writer/reader so popularity is not an issue here.  (The 7zip
algorithm and source code are both LGPL.)

In general (non-FITS), I expect that if people want files that are
provided in a certain compressed format then they will install the
decompressor (which is how I got onto 7zip in the first place).  bzip2
installation is trivial, there is a Debian package and presumably RPMs
etc.

>5) stability across a range of data sets - Even good ol' gzip varies  
>quite a bit in compression ratio from one file to the next.  For  
>example, the average gzip compression ratio over two years of NOAO  
>Mosaic II data is 0.586 +/- 0.0449.  Four and a half percent (1- 

>From the figures I sent yesterday it looks like 7zip gets an extra 25%
over gzip for binary data and decompresses at between x1 and x2 the
elapsed time.

For the freedb database 7zip compresses 30% better than gzip and
decompresses *faster* than gzip in elapsed time.  The 11-fold increase
in compression time would be amortized via repeated accesses to the
database.  

The bottom line is to use the technology that best suits the task.

Mark Calabretta
ATNF