[fitsmime] Thoughts on FITS MIME types.

[Clive Davenhall]

9/12/02.

Thanks for the first draft of the FITS MIME type RFC.  I thought that the
draft was a good start.  Appended below are some detailed comments, some of
which overlap with points that others have made.

Clive.

-----------------------------------------------------------------------------

        Comments on `MIME Sub-type Registrations for FITS', version:
           Id: rfcFITS.txt,v 1.29 2002/12/03 10:00:06 sla Exp $

First a few preliminaries.

* Somewhere in Section 3, probably in the introduction (`3.0'), I thought
  that it might be be useful to mention that though the FITS standard is
  is primarily intended for, and used in, astronomy it is sometimes
  encountered elsewhere.  For example, some general-purpose image viewers,
  such as xv can read FITS files.  Also, FITS is described in at least one
  book about image formats: D.C. Kay and J.R. Levine's, `Graphics File
  Formats' (1995), second edition (Windcrest/McGraw-Hill: New York), see in
  particular Chapter 18, pp235-244 (still available from Amazon).

* The comment in Section 4 that there should be archival copies of the
  FITS standard in non-US sites is reasonable, but I don't know of any
  sites that currently host it.  Some institutions that immediately come
  to mind as places which we could reasonably approach with a view to
  asking them to host archival copies are:

  - Centre de Donnees astronomiques de Strasbourg (CDS), France are the
    obvious choice.

  - Dept. of Physics and Astronomy, University of Leicester, UK already
    mirror many of the HEASARC pages as part of their LEDAS service, so it
    might be relatively straightforward for them to mirror the FITS pages
    from Goddard too.

  - Rutherford Appleton Lab (RAL), UK,

  - Canadian Astronomy Data Center (CADC).

* If compression using gzip or compress does not require a special MIME
  type, but rather an entry for the content-encoding (Section 4.6) then
  why does compression using hcompress need its own MIME type (Section
  4.2).  Should not hcompression also be handled by a content-encoding
  entry?  Or an I missing something?

Turning to the substance of the RFC, the MIME types needed to specify
FITS files, I'm not sure that there is much advantage in having lots of
different, specialised MIME types for different flavours of FITS files.

The possible combinations of different extensions are too many to
accommodate in any reasonable list of MIME types of reasonable length.
Many of the FITS files produced in practice won't precisely fit any of the
types and the programmer will be left trying to shoe-horn his file into the
nearest type.  Further, it is not obvious that the multiple types offer
significant advantages to the browser or client receiving the FITS file.
Ultimately all that can be done with a FITS file is to attempt to read
it using a program that understands FITS files, and either this program
will be able to understand the FITS file or it won't.  Either way, having
a broad description of the file in the MIME type won't help much.

General-purpose FITS readers, such as fv, which are the sort of application
most likely to be important in the VO context, will never be able to
understand purely local conventions for storing information in FITS files.
Unfortunately mosaics currently seem to be in this category (but see the
musings are the end of this message).  Sorting out a proper mechanism for
handling mosaics is a standardisation exercise that needs to happen in
the FITS community, not by defining MIME types.

I think that it would be better just to have 2 MIME types for FITS:

  (i)  image/fits
  (ii) application/fits
 
(i) is just a simple FITS file with an image (or array) in the PHDU.  I'm
ambivalent about whether (i) should be restricted to a FITS file with no
extensions or to allow the extensions associated with any WCS, as described
in the draft.

(ii) is any FITS file.  (That is (i) is a special case of (ii), and the
MIME type `application/fits' is valid for any file with MIME type
`image/fits'.)

In order to give the browser or client some additional hints about what is
in the FITS file, for example, to help it to pick a suitable application
to read the file, the following mechanism could be used.  The
`application/fits' MIME type could have the single, optional parameter:

  extensionlist

whose value is a comma-separated list of the extensions present in the
file.  Some examples might be:

Simple image in the PHDU:

  extensionlist='image'

A stack of images (maybe a data array, variance array and set of quality
flags):

  extensionlist='image,image,image'

An image in the PHDU followed by a binary table and an ASCII table:

  extensionlist='image,bintable,asctable'

The comma could also be used as a `place-holder' to indicate an empty PHDU.
So, for example, a FITS file that only contained a binary table would be:

  extensionlist=',bintable'

A further refinement would be use the word `array' rather than `image'
to describe array components.  This usage would be a departure from the
usual FITS tradition, but would be more precise, and, I think, preferable.
A simple FITS image would then become:

  extensionlist='array'

The advantages of this scheme are:

- only one parameter is required, and it can be fully specified,

- all the elements which can occur in the list that comprises the
  parameter's value can be specified; there is simply one for each of
  the standard extension types,

- there is a responsible authority (the FITS committees), who can invent
  new names if new extensions are approved,

- an accurate extensionlist can be constructed for any valid FITS file,
  and the procedure to do so is simple, unambiguous and amenable to
  automation.  Further, the elements in the extensionlist should help a
  browser or client to decide which FITS reader might be suitable for a
  given file.  For example there would be no point in passing a file with
  extensionlist=',bintable' to a reader that could not handle tables.

The extensionlist idea does, however, assume that the values of MIME
parameters can be reasonably long strings.

Mosaics
-------

This is a bit off-topic, but here are a few thoughts on mosaic images.
If the offsets between the individual images in the mosaic are specified
using bespoke, locally-defined keywords then a general-purpose viewer
has no chance of reconstructing the mosaic.  However, if instead the 
mosaic consisted of a stack of image extensions, each with its own WCS
for converting pixel positions into celestial coordinates, then it is
possible to imagine a FITS reader sophisticated enough to ruminate to
itself:

- this FITS file contains a stack of images,

- each has a WCS which transforms the pixel positions into the same
  celestial coordinate system,

- further, the individual images are pretty much adjacent to each other
  in this celestial coordinate system,

- aha!  I could work out the bounding box around these images and plot
  each image in its correct position inside the box.

But I'm not volunteering to write such a program, mind you!

-----------------------------------------------------------------------------
Clive Davenhall                                      Institute for Astronomy,
e-mail (internet, JANET): acd @ roe.ac.uk        Royal Observatory Edinburgh,
fax from within the UK:   0131-668-8416            Blackford Hill, Edinburgh,
fax from overseas:     +44-131-668-8416                    EH9 3HJ, Scotland.