[fitsmime] FITS MIME Considerations

[Lucio Chiappetti]

I apologize if these comments are too naive. Definitely they do not
represent the position of a project, but just of an user (although I had
some part a few years ago in an early AVO prototype
[http://terra.bo.cnr.it/avo/] and we encountered there the issue of
"distributing" FITS (image) files and somehow set it aside because of a
lack of a standard and a SUPPORT to such standard.

Let me ask a preliminary question (which probably many of the participants
to this FITS MIME idea have already answered, but has not been spelled out
explicitly).  Who/what (i.e. which person and which piece of s/w) will
need or benefit of a FITS MIME Content-Type ?

As far as I see, a MIME Content-Type is used mainly by two kind of
applications :

 1) mail clients which receive a data file as a MIME attachment to
    an e-mail message (and I believe this will not be the primary
    usage for FITS ... surelpy we won't encourage people to send large
    FITS files by e-mail cluttering spool areas which are often
    subject to quota or size limitations just to discourage this
    improper usage of e-mail)

 2) web browsers which use the MIME information to call a "viewer"
    this will be the majoritary use

    One could add perhaps a third way ...

 3) dedicated "network viewers" which connect directly via the http
    protocol to a server "giving away" FITS files

In all cases (but particularly in (2) and (3)) this requires the
cooperation of the server in declaring the Content-Type (for an httpd
server based on some AddType directive or mime.types file).

And it requires a suitable arrangement on the client side (a "suitable
arrangement" in most cases could just be "for an unknown Content-Type
offer to Save to Disk"  which is what most plain mail clients and browsers
do anyhow).

To have more useful actions, requires SUPPORT to the (new or unusual, e.g.
FITS) Content-Type from both the server and client sides.

Support from the client side can be arranged on an USER basis, e.g. adding
entries to one's .mailcap file. I'll do it sometimes (e.g. in my pine
e-mail client on Unix I configured to read MS-Word files in plain ASCII
via the "catdoc" utility :-) ).

But the common feeling is that asking to configure one's browser is "too
much" for the generic user. At least that was the feeling in the case of
our early AVO project, and it has still been the case recently in some
other more dedicated contexts.  A colleague comes to me and says "how can
the generic user automatically do this to such a file" and I say "he has
to place this in his own .mailcap" and he says "no, that's too much,
nobody will do it".

Therefore we need some WIDESPREAD SUPPORT at browser level (in case (2)
above ... case (3) it is paradoxically easier, because the average generic
user will probably agree to install a dedicated, pre-canned piece of
software) for some common actions to be taken in case of FITS files (which
cases I'll tell below).

We'd also need some WIDESPREAD SUPPORT at server level. Ideally what one'd
want would be to write an HTML page on one's server containing something
like

  <A HREF=myfile.mytype>This is a FITS file</a>

and having the user at the other end in the browser click on it and have
the appropriate action (view a FITS image, view as text a FITS table, save
to disk a generic FITS file)

So far to achieve only the last action (i.e. force the user to save the
file to disk as binary) I've used a construct via a CGI script :

  <A HREF=/cgi-bin/downloadgw?myfile.mytype>This is a FITS file</a>

with "downloadgw" being a silly script issuing a dummy Content-Type

echo "Content-type: application/x-data"
/bin/echo
/bin/cat $argv

We could have done it adding another non-standard MIME type, but since
there were none registered for FITS we just used the above trick to force
a binary Save to Disk.

What we'd need is to have a mechanism at server level which issues the
appropriate Content-Type header without the need of a CGI.

Note that I'd used "myfile.mytype" above, and not "myfile.fits", because I
consider an useless and annoying limitation the fact to use extension
".fits" (I'd prefer to use ".image", ".spectrum", ".mission_type" or
whatever which makes reference to the INTENDED USAGE).

In principle it won't be that difficult to define rules which tell some
basic kinds of FITS file as it is done in the Unix "magic" files (if we
content with recognising a generic FITS file, it's enough to look for
SIMPLE=T) ... what is unclear to me it's the level of support to "magic"
files by http servers (I see that one of my Apache conf directory contains
a magic file, but have no idea how it uses it). The mechanism is anyhow
superior to the mime.types approach.


What we'd need at the client level is a set of SUPPORTED VIEWERS (maybe
imbedded). In this respect these are the actions I'd expect :

 (a) for a "plain image" to display it, or to present a choice between
     displaying and saving to disk. Definitely we do not want that
     the fact a FITS MIME type causes an automatic display, forbid or
     make difficult to retrieve the file as binary ! Alhough the
     "click vs shift-click" paradigm can be OK.

 (b) for a "plain table" to view it as an HTML formatted table (many
     of us have written such HTML FITS viewers as CGI scripts, but
     it would be handier to have this invoked on the fly)

 (c) for "any other file" to be able to save it to disk.

[a] What do I mean as a "plain image" ?

    a plain image is definitely a NAXIS=2 image, with or without
    extensions required by WCS conventions. I suppose that the
    associated generic viewer should at least render the pixel image. It
    could make use of the WCS if it recognizes it. It should ignore
    extensions which does not recognise, but allow to save the entire
    file on disk.  And possibly should allow a way to display the
    file header.

    a NAXIS=1 image could also be accepted as a plain image (without
    resorting to the idiom I do not like NAXIS=2 NAXIS1=n NAXIS2=1)

    a NAXIS>=3 image could also be accepted, but the default action
    would be to render its first plane (more sophisticated viewers
    could prompt the user for the plane to display)

    a FITS file composed exclusivley by IMAGE extensions could also
    be handled in a way to prompt for the extension to display

    I doubt however whether the latter three cases require each one
    a different MIME type image/fits, application/fits-image1d,
    application/fits-image3d, application/fits-imageset ... or one
    could leave it as "additional non-default actions" to the
    viewer.  After all the viewer will have access to the full FITS
    header !

    I'm not deep enough into MIME to tell a preference between a type
    of image/fits and one of application/fits-image. But probably
    there should be only one of them !

[b] What do I mean by a "plain table" ?

    a plain table is a FITS file composed by a null primary HDU followed
    by a single TABLE extension (although my personal opinion is that
    ASCII tables should be superseded by BINTABLEs they may be still
    in use at catalogue sites).
    The generic viewer should render it either as an HTML formatted
    table, or as the ASCII table records, newline terminated and
    bracketed by <pre></pre>

    a plain table is also a FITS file composed by a null primary HDU
    followed by a single BINTABLE extension.
    Despite of the very different interpretations that the software
    for which the table is intended can assume (e.g. event files,
    response matrices, etc.) it should always make sense to "inspect"
    the table content and present it as an HTML formatted table
    (although some "cells" may be "clickable objects" like e.g. all
    those in a multi-dimensional column).

    There are CGI's which are used to inspect binary tables that
    way. Of course one could always decide that the viewer shows
    only the simplest things, and displays "unsupported" in the
    other cases.

    The table viewer too should allow saving to disk the entire file

    If a file is composed only by a number of BINTABLE extensions,
    a viewer could prompt for the extention to be displayed and
    perform as for a "plain table".

    As for the case [a] above, I'm not deep enough into MIME to tell the
    name for the MIME type. Should one distinguish
    application/fits-table and application/fits-bintable, or just use
    a generic wrapper (application/fits-table or even text/fits !) and
    let the viewer (which is or should be FULLY FITS CAPABLE) to solve
    the difference ?

In conclusion, from the formal point of view I agree with the statement
by William Thompson :

On Fri, 6 Dec 2002, William Thompson wrote:

> MIME types should also be restricted to only those types which have
> been officially recognized by the IAU, i.e. single HDU, random groups,
> and the extensions IMAGE, TABLE, and BINTABLE.  Conventions used by
> specific projects should not be incorporated until they're recognized
> by the IAU.

This means that image/fits-hcompress (whose reason AS DIFFERENT FROM A
compression encoding should be clear to everybody) should wait so far.

For the rest I consider three basic types

  images : image/fits or application/fits-image

  tables : text/fits (!) or application/fits-table and/or
           application/fits-bintable

  generic : application/fits

This does not mean I'm against random groups, only, since I do not know or
use them, I abstain.

However, while I won't see a proliferation beyond this, I have no strong
objections versus a simplified scheme with images on one hand, and any
other generic FITS files on the other.

----------------------------------------------------------------------------
Lucio Chiappetti - IASF/CNR - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.mi.iasf.cnr.it/~lucio/personal.html
----------------------------------------------------------------------------
Berlusconi's contract with Italians - Explained!  (adapted from Usenet 1996)
Contract: verb
1) To shrink or reduce in size - the economy contracted
2) To become infected - he contracted pneumonia when they stopped my welfare
----------------------------------------------------------------------------