compressed fits format

[Barry M. Schlesinger]

If other groups are developing compression algorithms, this would be a
good time to speak up, so that the community can work out common 
standards. 

In article <54jaio$2q8 at rossini.obs-nice.fr>, Alain MAURY <maury at ocar01.obs-azur.fr> writes...
> ...
>Is there any standard for compressed fits files ?.

There is no standard for compression within FITS and no standard 
compression algorithm.  The extension type name COMPRESS has been 
reserved for a future compressed FITS data type but the details of 
this type have not been worked out.  Warnock et al. have distributed a 
preliminary proposal, which is available at 
http://fits.cv.nrao.edu/documents/drafts/compression.*, where * is 
either tex (LaTeX) or ps (PostScript).  I suggest reading this paper 
to see what some previous thoughts on this subject have been.  
Individual installations are experimenting with local schemes. 

>I am about to include two modes of compression on our images, one
>with a lossless algorithm ( storage of the difference between 2 16
>bits pixels on a single byte ... and a lossy one after the one used for
>the digitised sky survey ( using H transform ). I ... 
> would like to know is there are standard FITS keywords
>describing these compression ( COMPRESS = T ? )
>We decided it was much better to perform real time processing of the
>data, then compress it by a factor of 10 and store it on CD ROMs ( which
>would then hold about a week of observations ) rather than to store
>everything on tape which cannot be read after 3 years... and would
>prefer to conform to some kind of standard if there is such a thing.

Such data could be included in a FITS file only as a nonstandard 
conforming extension.  The BITPIX, PCOUNT, GCOUNT, NAXIS and NAXISn 
keywords would have to have values that provided the size of the data 
area, as required by the generalized extensions agreement.  A separate 
extension type name would have to be defined for each kind of data, 
e.g., DIFFCOMP for the first algorithm and SURVCOMP for the second, and 
registered with the IAU FITS Working Group.  To conform to the rules, 
the names should be registered before the files are created. (Note: 
These are the first names that came off the top of my head; I am not
necessarily recommending them.)  There could then be software that
would recognize these names as signifying FITS images compressed in
the appropriate manner.  Other software would just use the information
in the header to skip the HDUs containing data compressed in these
ways.  

				Barry Schlesinger
				FITS Support Office
				GSFC/ADF