[fitsbits] CFITSIO and Compressed Images

[William Pence]

          FITS Compressed Image Support in CFITSIO

The new v2.036 version of the CFITSIO library for reading and writing
FITS files has been released, and one of the main new features
contained in it is trial support for the FITS image compression scheme
that was presented at the ADASS'99 meeting in Hawaii by Rick White,
Perry Greenfield, William Pence, and Doug Tody.  The CFITSIO software
is available from

http://heasarc.gsfc.nasa.gov/fitsio

and the latest draft of the image compression proposal is at

http://legacy.gsfc.nasa.gov/docs/heasarc/fits/compress/compress_image.html

or in postscript format at:

http://legacy.gsfc.nasa.gov/docs/heasarc/fits/compress/compress_image.ps

Under this compression scheme the FITS image (with any number of
dimensions) is compressed using one of several different algorithms,
and the resulting compressed byte stream is written into a
variable-length array in a FITS binary table.  Optionally, the image
may be 'tiled' into a rectangular grid of subimages, and each subimage
is compressed and stored in a separate row of the binary table.  This
'tiled' mode of storage provides faster random access and reduced
memory requirements for accessing very large compressed images.

This release of CFITSIO currently supports 3 compression methods: GZIP,
RICE, and IRAF/PLIO.  In principle it should be easy to add support for
other compression algorithms. The IRAF/PLIO algorithm is intended for
use in image pixel masks and not for general image compression.  In all
3 of these compression methods, integer images are compressed without
any loss of information, but floating point images are first quantized
into discrete integer values before being compressed.  This results in
a small but usually insignificant loss of information in the compressed
image as compared to the original floating point image.

The source files for 2 sample programs that demonstrate this
compression technique are included in this release of CFITSIO.  The
programs are called compress_fits.c and uncompress_fits.c.  The first
program will compress any existing FITS image using a choice of
compression algorithm and tiling pattern.  The second program will
uncompress the image, restoring it to the original format.  These
programs may be compiled and built on Unix platforms simply by typing
'make compress_fits' and 'make uncompress_fits' after building the
CFITSIO library itself.  Entering the name of these programs on the
command line, without any arguments, will generate a short listing
describing how to use the programs.

All the interface routines in the CFITSIO library that read FITS images
now transparently support this compressed image format.  Any programs
that use CFITSIO to read FITS images are able to read these compressed
images (which are actually stored in FITS binary tables), with little
or no modification.  Typically the only change that will be needed is
to call a new CFITSIO interface routine to get the datatype and size of
the image, rather than reading the values of the BITPIX and NAXISn
keywords directly (which now of course describe the format of the
binary table, not the original image).

Preliminary tests of the efficiency of these routines are encouraging.
Compression factors ranging from 3 - 26 were achieved, depending on the
content of file and on the compression algorithm.  In these limited
tests, the RICE algorithm generally performed better than the GZIP
algorithm.  The time required to read the compressed images was not
much greater, if any, than reading the uncompressed image;  The
computational overhead required to uncompress the image is at least
partially offset by the reduced number of bytes that have to be read
from disk.

This version of CFITSIO only supports reading compressed images; it is
not possible yet to modify or write to a compressed image.  Currently
there is only one compression routine provided in CFITSIO and it
compresses an entire FITS image; it is used in the compress_fits.c
sample program.  It is anticipated that future versions of CFITSIO will
support incremental writing or modifying the pixel values in a
compressed image.

This release of the compression software in CFITSIO should be
considered provisional; it is possible (although we hope unlikely) that
the format of the compressed image will change in later versions of the
software after we gain more experience.  We encourage others to
experiment with this new compression facility, and send us any comments
or suggestions about the software.

Bill Pence, NASA/GSFC
Rick White. STScI
Perry Greenfield. STScI
Doug Tody, NOAO