From dwells at nrao.edu  Mon Dec  2 11:51:08 2002
From: dwells at nrao.edu (Don Wells)
Date: Mon, 2 Dec 2002 11:51:08 -0500
Subject: [fitsmime] Test#2
Message-ID: <15851.36732.967429.40976@fits.cv.nrao.edu>

This is a test of the new FITS MIME exploder.

Address <fitsmime at nrao.edu> is not yet in the alias table,
so <fitsmime at listmgr.cv.nrao.edu> must be used.

-Don
-- 
  Donald C. Wells      Scientist - GBT Project        dwells at nrao.edu
                    http://www.cv.nrao.edu/~dwells
  National Radio Astronomy Observatory                +1-434-296-0277
  520 Edgemont Road,   Charlottesville, Virginia       22903-2475 USA
       (DCW is often in Green Bank, West Virginia, at +1-304-456-2146)


From dwells at nrao.edu  Tue Dec  3 08:15:34 2002
From: dwells at nrao.edu (Don Wells)
Date: Tue, 3 Dec 2002 08:15:34 -0500
Subject: [fitsmime] Test#3
Message-ID: <15852.44662.6292.685935@fits.cv.nrao.edu>

This message tests whether <nrao.edu> knows the alias <fitsmime>.
-- 
  Donald C. Wells      Scientist - GBT Project        dwells at nrao.edu
                    http://www.cv.nrao.edu/~dwells
  National Radio Astronomy Observatory                +1-434-296-0277
  520 Edgemont Road,   Charlottesville, Virginia       22903-2475 USA
       (DCW is often in Green Bank, West Virginia, at +1-304-456-2146)


From sla at ucolick.org  Tue Dec  3 15:39:18 2002
From: sla at ucolick.org (Steve Allen)
Date: Tue, 3 Dec 2002 12:39:18 -0800
Subject: [fitsmime] FITS MIME welcome and initial agenda
Message-ID: <20021203203918.GA13853@ucolick.org>

Greetings and welcome to all on the fitsmime list.

Don Wells has arranged for this list to be hosted at NRAO, and the
initial membership is composed of a very small handful of folks whom I
have contacted in the past few weeks.

The initiative for this effort came out of a query from Bill Joye
who has been implementing various VO features in ds9.  He has
encountered a wide variety of different MIME types already being
used by various astronomical archive servers.  The time seems ripe
to begin making official the marriage between FITS and the internet.

The list server should have informed everyone of the web interface
for managing subscriptions.  If not, it is at
http://listmgr.cv.nrao.edu/mailman/listinfo/fitsmime

There are two items on my initial agenda:

Membership

I believe the existence of this list should be publicly announced
to the fitsbits mail list and to various VO mail lists.
Also, there are almost certainly other individuals who should
be specifically invited into the mail list.

Please feel free to invite others onto this list.

Draft Documents

In order to register MIME types officially the FITS community will
have to submit a registration request to the IANA in tandem with an
RFC to the IETF/IESG.  The details of this process, a draft of the
combined RFC and registration requests are visible on this web page

http://www.ucolick.org/~sla/fits/mime/

There is also a lot of my own commentary on issues that I think this
mailing list will have to consider before the draft can be submitted
for approval by FITS or internet committees.

I am not certain how much discussion on the content of the drafts
should begin in advance of the general announcement of this list.
I am open to suggestions on that matter.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From joye at head-cfa.cfa.harvard.edu  Tue Dec  3 17:10:36 2002
From: joye at head-cfa.cfa.harvard.edu (William Joye)
Date: Tue, 3 Dec 2002 17:10:36 -0500 (EST)
Subject: [fitsmime] FITS Mime-Types
Message-ID: <200212032210.gB3MAai14680@head-cfa.cfa.harvard.edu>

Well, since I'm the one who started the recent fuss, I'll begin with what I 
need, and what I've found out so far...

As author of ds9, I need to know the type of file a web site is trying to 
download into ds9, and if its compressed or not, and how.

I've reviewed Steve's RFC and agree with it 100%. The problem now is to get it 
submitted and to get the various archive sites to conform.

Currently, the various archive sites are using a host of mime-types and 
content-encoding strings. This is what I've encountered so far:

mime-type			what			where
---------			----			-----
image/fits			fits			RFC
image/fits-hcompress		hcompress'd		RFC
application/fits-image		image			RFC
application/fits-table		bin table		RFC
application/fits-group					RFC

image/x-fits			uncompressed image	strasbg/uk
binary/x-fits			uncompressed bin table	strasbg/uk
application/x-fits		uncompressed image	ESO DSS server

image/x-gfits			gzip'd image		strasbg/uk
binary/x-gfits			gzip'd bintable		strasbg/uk
image/gz-fits			gzip'd image		?
display/gz-fits			gzip'd image		ESO DSS server

image/x-cfits			compress'd image	strasbg/uk
binary/x-cfits			compress'd bin table	strasbg/uk

image/x-hfits			hcompress'd image	strasbg/uk
binary/x-hfits			hcompress'd bin table	strasbg/uk
image/x-fits-h			hcompress'd image	MAST

and

content-encoding		where
----------------		-----
gzip				RFC
compress			RFC
pack				RFC
x-gzip				MAST/NED

currently in ds9 2.2.1, I support all the above mime-types, but no 
content-encoding. Starting with version 2.3b1, ds9 will fully support both the 
RFC mime-types/content-encoding, and backward compatibility with older 
'nonstandard' mime-types, as long as they are still in use.

Bill Joye
wjoye at cfa.harvard.edu
Smithsonian Astrophysical Observatory
Harvard-Smithsonian Center for Astrophysics



From joye at head-cfa.cfa.harvard.edu  Fri Dec  6 11:12:15 2002
From: joye at head-cfa.cfa.harvard.edu (William Joye)
Date: Fri, 6 Dec 2002 11:12:15 -0500 (EST)
Subject: [fitsmime] FITS Mime-Types for Mosaics
Message-ID: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu>

It has come to my attention that we have a need to handle a new case, one where 
there is one FITS file, with many extensions, that together form a mosaic image, 
and is to be loaded as such.

My first pass at this problem would be a new mime-type (since it is not an 
encoding issue) of something like:

image/fits-mosaic

this would fit in well with the existing proposed mime-types... comments?

Bill Joye
wjoye at cfa.harvard.edu
Smithsonian Astrophysical Observatory
Harvard-Smithsonian Center for Astrophysics



From thompson at orpheus.nascom.nasa.gov  Fri Dec  6 11:14:38 2002
From: thompson at orpheus.nascom.nasa.gov (William Thompson)
Date: Fri, 06 Dec 2002 11:14:38 -0500
Subject: [fitsmime] FITS mime types
Message-ID: <3DF0CCEE.12E4A2C@orpheus.nascom.nasa.gov>

Just to chime in, I've been using "image/x-fits" for FITS files containing 2D
images (i.e. NAXIS=2), and "application/x-fits" for FITS files in general.  The
general convention around here has been to use the extension ".fts" for image
files, and either ".fit" or ".fits" for more general FITS files.  Hence, my
mime.types file contains

application/x-fits		fits fit
image/x-fits			fts

William Thompson


From thompson at orpheus.nascom.nasa.gov  Fri Dec  6 11:55:13 2002
From: thompson at orpheus.nascom.nasa.gov (William Thompson)
Date: Fri, 06 Dec 2002 11:55:13 -0500
Subject: [fitsmime] FITS Mime-Types for Mosaics
References: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu>
Message-ID: <3DF0D671.38CC9FA4@orpheus.nascom.nasa.gov>

My own thoughts on this are that "image/fits" and any "image/fits-..." should be
reserved for FITS images which one could expect to be loaded into a generic
image viewing program (such as XV or LVIEW) which also handles other image types
such as GIF and JPEG.  To my mind, scientific processing, such as forming a
mosaic out of a series of images, is not appropriate for the "image/..." MIME
type.

MIME types should also be restricted to only those types which have been
officially recognized by the IAU, i.e. single HDU, random groups, and the
extensions IMAGE, TABLE, and BINTABLE.  Conventions used by specific projects
should not be incorporated until they're recognized by the IAU.

Personally, I think that the concentration should be on defining two mime types,
one for 2D images (image/fits), and another as a catch-all for FITS as a whole
(application/fits).  I don't really see much utility in types such as
"application/fits-bintable" since the internal structure can be so different
from file to file.  My binary tables are probably completely different from
yours.  (On the other hand, I could see that the MIME types
"application/fits-group", "application/fits-table", and "application/fits-image"
might have some utility.)

I don't know how I would handle a file with the mime type
"application/fits-mosaic", unless there were a universally recognized way of
encoding such data.  There's certainly nothing in the FITS documentation listing
such a standard.  I can personally think of three different ways of encoding
mosaics within FITS files.

William Thompson


William Joye wrote:
> 
> It has come to my attention that we have a need to handle a new case, one where
> there is one FITS file, with many extensions, that together form a mosaic image,
> and is to be loaded as such.
> 
> My first pass at this problem would be a new mime-type (since it is not an
> encoding issue) of something like:
> 
> image/fits-mosaic
> 
> this would fit in well with the existing proposed mime-types... comments?
> 
> Bill Joye
> wjoye at cfa.harvard.edu
> Smithsonian Astrophysical Observatory
> Harvard-Smithsonian Center for Astrophysics
> 
> _______________________________________________
> fitsmime mailing list
> fitsmime at listmgr.cv.nrao.edu
> http://listmgr.cv.nrao.edu/mailman/listinfo/fitsmime


From sla at ucolick.org  Fri Dec  6 12:07:22 2002
From: sla at ucolick.org (Steve Allen)
Date: Fri, 6 Dec 2002 09:07:22 -0800
Subject: [fitsmime] FITS Mime-Types for Mosaics
In-Reply-To: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu>
References: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu>
Message-ID: <20021206170722.GA28941@ucolick.org>

On Fri 2002-12-06T11:12:15 -0500, William Joye hath writ:

> image/fits-mosaic
>
> this would fit in well with the existing proposed mime-types... comments?

Being a producer of FITS files from a mosaic detector, I agree
wholeheartedly that there should be such a MIME type.  In the same
breath, however, I have to point out that there are no official
conventions and, indeed, no unofficial conventions which describe the
properties of a mosaic file.

Bill and I are intimately familiar with this.  We have been
collaborating informally for several years to ensure that ds9 can
consume mosaic files both from NOAO (which has the first CCD mosaic
with widely-distributed files, and which is understood by IRAF)
and from my own code for the DEIMOS mosaic (which conforms with
NOAO only to the extent required by ds9, but aspires to use an
entirely different mechanism for describing the mosaic layout
that is based on the WCS papers).


After rewriting 29 drafts of the RFC I have come to the opinion
that, within the scope of FITS convention and agreement, there
are only two kinds of FITS files, and therefore can only be
two MIME types:

application/fits
    This would be the default MIME type for FITS files which are not
    classified by their creator.

image/fits
    This would be the MIME type for FITS files whose creator is
    willing to assert that they are principally intended to
    communicate the content of the image in the PHDU.

Any other MIME types would require more conventions and agreements
within the FITS community.  I believe that more such conventions and
agreements would be a good thing.  I also believe that they will not
happen anytime soon, and that the need for these two basic MIME types
for FITS is great enough to proceed now.

I am not convinced that the scope of this discussion should stray into
the field of getting more conventions and agreements from the FITS
community, but that depends on the urgency of the timescale to
produce the (first) RFC.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From joye at head-cfa.cfa.harvard.edu  Fri Dec  6 13:50:01 2002
From: joye at head-cfa.cfa.harvard.edu (William Joye)
Date: Fri, 6 Dec 2002 13:50:01 -0500 (EST)
Subject: [fitsmime] FITS Mime-Types for Mosaics
Message-ID: <200212061850.gB6Io1K00521@head-cfa.cfa.harvard.edu>

Valid point. I think it is far more urgent to produce the RFC than to think 
about new mime-types. I only put this out to the community to start the 
discussion process. But lets not hold up the RFC on account of this.

One other idea. Is there a way we can use the HTTP Meta data field to indicate 
an interpretation of the FITS data (i.e. it is a events bin table, or it is a 
mosaic image...?) this would get around the problems of formally defining new 
mime-types...

I like the two proposed mime-types, simple, straight to the point, no ambiguity.

<snip>

>After rewriting 29 drafts of the RFC I have come to the opinion
>that, within the scope of FITS convention and agreement, there
>are only two kinds of FITS files, and therefore can only be
>two MIME types:
>
>application/fits
>    This would be the default MIME type for FITS files which are not
>    classified by their creator.
>
>image/fits
>    This would be the MIME type for FITS files whose creator is
>    willing to assert that they are principally intended to
>    communicate the content of the image in the PHDU.
>
>Any other MIME types would require more conventions and agreements
>within the FITS community.  I believe that more such conventions and
>agreements would be a good thing.  I also believe that they will not
>happen anytime soon, and that the need for these two basic MIME types
>for FITS is great enough to proceed now.
>
>I am not convinced that the scope of this discussion should stray into
>the field of getting more conventions and agreements from the FITS
>community, but that depends on the urgency of the timescale to
>produce the (first) RFC.
>
>--
>Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
>sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
>PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93
>_______________________________________________
>fitsmime mailing list
>fitsmime at listmgr.cv.nrao.edu
>http://listmgr.cv.nrao.edu/mailman/listinfo/fitsmime

Bill Joye
wjoye at cfa.harvard.edu
Smithsonian Astrophysical Observatory
Harvard-Smithsonian Center for Astrophysics



From sla at ucolick.org  Fri Dec  6 14:04:20 2002
From: sla at ucolick.org (Steve Allen)
Date: Fri, 6 Dec 2002 11:04:20 -0800
Subject: [fitsmime] FITS Mime-Types for Mosaics
In-Reply-To: <200212061850.gB6Io1K00521@head-cfa.cfa.harvard.edu>
References: <200212061850.gB6Io1K00521@head-cfa.cfa.harvard.edu>
Message-ID: <20021206190420.GB31039@ucolick.org>

On Fri 2002-12-06T13:50:01 -0500, William Joye hath writ:
> One other idea. Is there a way we can use the HTTP Meta data field to indicate
> an interpretation of the FITS data (i.e. it is a events bin table, or it is a
> mosaic image...?) this would get around the problems of formally defining new
> mime-types...

That is the question that I begain to raise in my ruminations about
the "Optional parameters" types of parameter fields which are sent
along with the MIME media type.

I do not yet feel familiar enough with the spectrum of usage of
"Optional parameters" in other MIME types to be sure of what we can
and cannot do with them.  In particular, I don't know whether we can
or should submit a RFC which asserts that
	there may be some optional parameters but we don't yet know
	their names or admissible values.
or
	there are some optional parameters, which we have named
	and listed some admissible values, but for which we may
	create more admissible values later.

It is my sense that either of these two possibilities requires FITS to
have some sort of naming authority doing a job akin to that of the
IANA, and that the RFC would have to give reference to at least one
website which publishes the decisions of that naming authority.  The
same sort of naming authority would probably also be able to serve as
a registry of information describing how to interpret various local
conventions of FITS files.  But this all presumes both a technical
mechanism (perhaps some sort of XML as the language?) and a social
mechanism that don't exist.

(Along those lines but tangential, I noted with some horror that I
could only find the FITS standard online from websites within the
United States.)

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From sla at ucolick.org  Fri Dec  6 14:17:20 2002
From: sla at ucolick.org (Steve Allen)
Date: Fri, 6 Dec 2002 11:17:20 -0800
Subject: [fitsmime] FITS Mime-Types for Mosaics
In-Reply-To: <20021206190420.GB31039@ucolick.org>
References: <200212061850.gB6Io1K00521@head-cfa.cfa.harvard.edu> <20021206190420.GB31039@ucolick.org>
Message-ID: <20021206191720.GA31529@ucolick.org>

On Fri 2002-12-06T11:04:20 -0800, Steve Allen hath writ:
> I do not yet feel familiar enough with the spectrum of usage of
> "Optional parameters" in other MIME types to be sure of what we can
> and cannot do with them.

(Pardon me for not mentioning this in the previous note.)


From dwells at nrao.edu  Fri Dec  6 15:09:59 2002
From: dwells at nrao.edu (Don Wells)
Date: Fri, 6 Dec 2002 15:09:59 -0500
Subject: [fitsmime] FITS Mime-Types for Mosaics
In-Reply-To: <3DF0D671.38CC9FA4@orpheus.nascom.nasa.gov>
References: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu>
	<3DF0D671.38CC9FA4@orpheus.nascom.nasa.gov>
Message-ID: <15857.1047.743018.450069@fits.cv.nrao.edu>

William Thompson writes:
 > My own thoughts on this are that "image/fits" and any
 > "image/fits-..." should be reserved for FITS images which one could
 > expect to be loaded into a generic image viewing program (such as
 > XV or LVIEW) which also handles other image types such as GIF and
 > JPEG.  To my mind, scientific processing, such as forming a mosaic
 > out of a series of images, is not appropriate for the "image/..."
 > MIME type.

I agree. The goal of registering the toplevel type 'image/fits' with
the IETF/IESG is to encourage the non-astronomy imaging tools to
support our most common type of image data.  (A surprising number of
them support us already.)

Note that an 'image/fits' file may have extensions, such as tables of
detected sources, but the idea is that 'image/fits' means that the
primary content of the file is an N-dimensional matrix in the PHDU.

 > MIME types should also be restricted to only those types which have
 > been officially recognized by the IAU, i.e. single HDU, random
 > groups, and the extensions IMAGE, TABLE, and BINTABLE.  Conventions
 > used by specific projects should not be incorporated until they're
 > recognized by the IAU.

Again, I agree. That was approximately my reasoning some years ago
when I proposed the short list of 'application/fits-*' types.

 > ..one for 2D images (image/fits)..

N-dimensional, not just 2-D.  3-D imagery are common, and will be even
more common in the future.  (Most computer systems sold today have 3-D
hardware acceleration -- the Dell computer on which I am entering this
text has an ATI 'Rage128' chipset, with software support on both Linux
and Windows.)

 > ..  I don't really see much utility in types such as
 > "application/fits-bintable" since the internal structure can be so
 > different from file to file..  (On the other hand, I could see that
 > the MIME types "application/fits-group", "application/fits-table",
 > and "application/fits-image" might have some utility.)..

I proposed the latter three categories with the idea that they would
have the following meanings:

* "application/fits-group" -- the primary content of the file is a
			      single random group structure, but there
			      may be extensions. 

* "application/fits-table" -- the primary content of the file is a
			      *set* of tables (TABLEs and/or BINTABLEs).

* "application/fits-image" -- the primary content of the file is a
			      *set* of IMAGE extensions, but there may
			      also be table extensions in the file.  

 > I don't know how I would handle a file with the mime type
 > "application/fits-mosaic", unless there were a universally
 > recognized way of encoding such data.

Because there is not yet an agreed convention for mosaics,
"application/fits-mosaic" is probably premature at this time, although
it may make sense in the future if/when such agreements are produced.

				 -=-

More generally, I wonder whether fine-grain classification of our
files might not be better achieved with a reserved keyword in the
primary header.  The keyword would have value strings coded according
to some convention.  Such a string could specify that an
'application/fits-image' file is an NOAO mosaic, or a DEIMOS mosaic,
or some other scheme for using a set of IMAGE extensions (plus table
extensions) to convey some sort of image data.  The same keyword, or
maybe an analogous keyword, would be used in 'application/fits-table'
files to say whether the content is a set of tables conveying an event
list from a high energy experiment, with the schema conforming to some
convention, or instead the content is a set of tables conveying radio
aperture synthesis fringe visibilities in conformance with the
FITS-IDI (Interferometer Data Interchange) convention. 

Usually the client-side implementation of MIME codes is that one
application code is associated with each code.  It seems to me that it
should be possible to code a simple program which could decide the
provenance and probable detailed type of a FITS file by examining the
headers, even in the absence of a keyword such as the one discussed in
the previous paragraph, and which could then decide which of a set of
application programs will be used to process each individual file.
The classification program would then invoke the proper application.
I.e., if a 'application/fits-table' file is an event list, invoke a
HEASARC or CHANDRA or XMM etc. application, and if it is fringe
visibilities, invoke Classic-AIPS or Miriad or GILDAS etc.  So, while
I think that a classification keyword might be a good thing, I don't
think that it is absolutely necessary in order to get effective
interchange, and I don't think that we need a big set of MIME codes
for all of the detailed types of FITS files. 

-Don
-- 
  Donald C. Wells      Scientist - GBT Project        dwells at nrao.edu
                    http://www.cv.nrao.edu/~dwells
  National Radio Astronomy Observatory                +1-434-296-0277
  520 Edgemont Road,   Charlottesville, Virginia       22903-2475 USA
       (DCW is often in Green Bank, West Virginia, at +1-304-456-2146)


From thompson at orpheus.nascom.nasa.gov  Fri Dec  6 15:30:02 2002
From: thompson at orpheus.nascom.nasa.gov (William Thompson)
Date: Fri, 06 Dec 2002 15:30:02 -0500
Subject: [fitsmime] FITS Mime-Types for Mosaics
References: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu>
		<3DF0D671.38CC9FA4@orpheus.nascom.nasa.gov> <15857.1047.743018.450069@fits.cv.nrao.edu>
Message-ID: <3DF108CA.859CC053@orpheus.nascom.nasa.gov>

I know that the word "image" is used differently in the FITS community than in
the rest of the world, but it appears that its meaning in the MIME community is
the more conventional non-FITS meaning.  All the image/xyz mime types listed
that I can recognize are certainly 2D images.  This is also implied by the
following text from RFC2046:

    (2)   image -- image data.  "Image" requires a display device
          (such as a graphical display, a graphics printer, or a
          FAX machine) to view the information. An initial
          subtype is defined for the widely-used image format
          JPEG. .  subtypes are defined for two widely-used image
          formats, jpeg and gif.

William Thompson


From sla at ucolick.org  Fri Dec  6 16:04:13 2002
From: sla at ucolick.org (Steve Allen)
Date: Fri, 6 Dec 2002 13:04:13 -0800
Subject: [fitsmime] FITS Mime-Types for Mosaics
In-Reply-To: <3DF108CA.859CC053@orpheus.nascom.nasa.gov>
References: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu> <3DF0D671.38CC9FA4@orpheus.nascom.nasa.gov> <15857.1047.743018.450069@fits.cv.nrao.edu> <3DF108CA.859CC053@orpheus.nascom.nasa.gov>
Message-ID: <20021206210413.GA918@ucolick.org>

On Fri 2002-12-06T15:30:02 -0500, William Thompson hath writ:

> that I can recognize are certainly 2D images.  This is also implied by the
> following text from RFC2046:
>
>     (2)   image -- image data.  "Image" requires a display device
>           (such as a graphical display, a graphics printer, or a
>           FAX machine) to view the information. An initial
>           subtype is defined for the widely-used image format
>           JPEG. .  subtypes are defined for two widely-used image
>           formats, jpeg and gif.

The IANA registry contains examples of "image" types which are
nothing like a jpeg or gif.

E.g., autocad files with 3-d vector models:

http://www.iana.org/assignments/media-types/image/vnd.dwg
http://www.iana.org/assignments/media-types/image/vnd.dxf
http://www.iana.org/assignments/media-types/image/vnd.svf

I think that's a broad enough precedent not to worry about
the dimensionality of a PHDU image.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From dwells at nrao.edu  Fri Dec  6 16:05:15 2002
From: dwells at nrao.edu (Don Wells)
Date: Fri, 6 Dec 2002 16:05:15 -0500
Subject: [fitsmime] FITS Mime-Types for Mosaics
In-Reply-To: <3DF108CA.859CC053@orpheus.nascom.nasa.gov>
References: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu>
	<3DF0D671.38CC9FA4@orpheus.nascom.nasa.gov>
	<15857.1047.743018.450069@fits.cv.nrao.edu>
	<3DF108CA.859CC053@orpheus.nascom.nasa.gov>
Message-ID: <15857.4363.314076.967112@fits.cv.nrao.edu>

William Thompson writes:
 > .. "image" .. in the MIME community [appears to mean] 2D images..

If we find that NAXIS != 2 is wrong for 'image/*', 
then maybe our proposal should be something like:

'image/fits'                -- NAXIS=2 image in the PHDU, plus optional
                               extensions

'application/fits-image1d'  -- NAXIS=1 in the PHDU, plus optional table
                               extensions

'application/fits-image3d'  -- NAXIS>=3 in the PHDU, plus optional
                               table extensions

'application/fits-imageset' -- a *set* of IMAGE extensions, plus optional
                               table extensions

-Don


From acd at roe.ac.uk  Mon Dec  9 12:03:16 2002
From: acd at roe.ac.uk (Clive Davenhall)
Date: Mon, 9 Dec 2002 17:03:16 +0000 (GMT)
Subject: [fitsmime] Thoughts on FITS MIME types.
Message-ID: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk>

9/12/02.

Thanks for the first draft of the FITS MIME type RFC.  I thought that the
draft was a good start.  Appended below are some detailed comments, some of
which overlap with points that others have made.

Clive.

-----------------------------------------------------------------------------

        Comments on `MIME Sub-type Registrations for FITS', version:
           Id: rfcFITS.txt,v 1.29 2002/12/03 10:00:06 sla Exp $

First a few preliminaries.

* Somewhere in Section 3, probably in the introduction (`3.0'), I thought
  that it might be be useful to mention that though the FITS standard is
  is primarily intended for, and used in, astronomy it is sometimes
  encountered elsewhere.  For example, some general-purpose image viewers,
  such as xv can read FITS files.  Also, FITS is described in at least one
  book about image formats: D.C. Kay and J.R. Levine's, `Graphics File
  Formats' (1995), second edition (Windcrest/McGraw-Hill: New York), see in
  particular Chapter 18, pp235-244 (still available from Amazon).

* The comment in Section 4 that there should be archival copies of the
  FITS standard in non-US sites is reasonable, but I don't know of any
  sites that currently host it.  Some institutions that immediately come
  to mind as places which we could reasonably approach with a view to
  asking them to host archival copies are:

  - Centre de Donnees astronomiques de Strasbourg (CDS), France are the
    obvious choice.

  - Dept. of Physics and Astronomy, University of Leicester, UK already
    mirror many of the HEASARC pages as part of their LEDAS service, so it
    might be relatively straightforward for them to mirror the FITS pages
    from Goddard too.

  - Rutherford Appleton Lab (RAL), UK,

  - Canadian Astronomy Data Center (CADC).

* If compression using gzip or compress does not require a special MIME
  type, but rather an entry for the content-encoding (Section 4.6) then
  why does compression using hcompress need its own MIME type (Section
  4.2).  Should not hcompression also be handled by a content-encoding
  entry?  Or an I missing something?

Turning to the substance of the RFC, the MIME types needed to specify
FITS files, I'm not sure that there is much advantage in having lots of
different, specialised MIME types for different flavours of FITS files.

The possible combinations of different extensions are too many to
accommodate in any reasonable list of MIME types of reasonable length.
Many of the FITS files produced in practice won't precisely fit any of the
types and the programmer will be left trying to shoe-horn his file into the
nearest type.  Further, it is not obvious that the multiple types offer
significant advantages to the browser or client receiving the FITS file.
Ultimately all that can be done with a FITS file is to attempt to read
it using a program that understands FITS files, and either this program
will be able to understand the FITS file or it won't.  Either way, having
a broad description of the file in the MIME type won't help much.

General-purpose FITS readers, such as fv, which are the sort of application
most likely to be important in the VO context, will never be able to
understand purely local conventions for storing information in FITS files.
Unfortunately mosaics currently seem to be in this category (but see the
musings are the end of this message).  Sorting out a proper mechanism for
handling mosaics is a standardisation exercise that needs to happen in
the FITS community, not by defining MIME types.

I think that it would be better just to have 2 MIME types for FITS:

  (i)  image/fits
  (ii) application/fits
 
(i) is just a simple FITS file with an image (or array) in the PHDU.  I'm
ambivalent about whether (i) should be restricted to a FITS file with no
extensions or to allow the extensions associated with any WCS, as described
in the draft.

(ii) is any FITS file.  (That is (i) is a special case of (ii), and the
MIME type `application/fits' is valid for any file with MIME type
`image/fits'.)

In order to give the browser or client some additional hints about what is
in the FITS file, for example, to help it to pick a suitable application
to read the file, the following mechanism could be used.  The
`application/fits' MIME type could have the single, optional parameter:

  extensionlist

whose value is a comma-separated list of the extensions present in the
file.  Some examples might be:

Simple image in the PHDU:

  extensionlist='image'

A stack of images (maybe a data array, variance array and set of quality
flags):

  extensionlist='image,image,image'

An image in the PHDU followed by a binary table and an ASCII table:

  extensionlist='image,bintable,asctable'

The comma could also be used as a `place-holder' to indicate an empty PHDU.
So, for example, a FITS file that only contained a binary table would be:

  extensionlist=',bintable'

A further refinement would be use the word `array' rather than `image'
to describe array components.  This usage would be a departure from the
usual FITS tradition, but would be more precise, and, I think, preferable.
A simple FITS image would then become:

  extensionlist='array'

The advantages of this scheme are:

- only one parameter is required, and it can be fully specified,

- all the elements which can occur in the list that comprises the
  parameter's value can be specified; there is simply one for each of
  the standard extension types,

- there is a responsible authority (the FITS committees), who can invent
  new names if new extensions are approved,

- an accurate extensionlist can be constructed for any valid FITS file,
  and the procedure to do so is simple, unambiguous and amenable to
  automation.  Further, the elements in the extensionlist should help a
  browser or client to decide which FITS reader might be suitable for a
  given file.  For example there would be no point in passing a file with
  extensionlist=',bintable' to a reader that could not handle tables.

The extensionlist idea does, however, assume that the values of MIME
parameters can be reasonably long strings.

Mosaics
-------

This is a bit off-topic, but here are a few thoughts on mosaic images.
If the offsets between the individual images in the mosaic are specified
using bespoke, locally-defined keywords then a general-purpose viewer
has no chance of reconstructing the mosaic.  However, if instead the 
mosaic consisted of a stack of image extensions, each with its own WCS
for converting pixel positions into celestial coordinates, then it is
possible to imagine a FITS reader sophisticated enough to ruminate to
itself:

- this FITS file contains a stack of images,

- each has a WCS which transforms the pixel positions into the same
  celestial coordinate system,

- further, the individual images are pretty much adjacent to each other
  in this celestial coordinate system,

- aha!  I could work out the bounding box around these images and plot
  each image in its correct position inside the box.

But I'm not volunteering to write such a program, mind you!

-----------------------------------------------------------------------------
Clive Davenhall                                      Institute for Astronomy,
e-mail (internet, JANET): acd @ roe.ac.uk        Royal Observatory Edinburgh,
fax from within the UK:   0131-668-8416            Blackford Hill, Edinburgh,
fax from overseas:     +44-131-668-8416                    EH9 3HJ, Scotland.




From sla at ucolick.org  Mon Dec  9 12:33:37 2002
From: sla at ucolick.org (Steve Allen)
Date: Mon, 9 Dec 2002 09:33:37 -0800
Subject: [fitsmime] FITS Mime-Types for Mosaics
In-Reply-To: <15857.1047.743018.450069@fits.cv.nrao.edu>
References: <200212061612.gB6GCFK22069@head-cfa.cfa.harvard.edu> <3DF0D671.38CC9FA4@orpheus.nascom.nasa.gov> <15857.1047.743018.450069@fits.cv.nrao.edu>
Message-ID: <20021209173337.GA14957@ucolick.org>

On Fri 2002-12-06T15:09:59 -0500, Don Wells hath writ:
> I proposed the latter three categories with the idea that they would
> have the following meanings:

> * "application/fits-table" -- the primary content of the file is a
>                             *set* of tables (TABLEs and/or BINTABLEs).
>
> * "application/fits-image" -- the primary content of the file is a
>                             *set* of IMAGE extensions, but there may
>                             also be table extensions in the file.

I am not convinced of the utility of fits-table and fits-image
vs. simply fits.  If I encounter a fits-table file I do not
know whether to expect it to be an X-ray event list (which is
really an image) or a dump of somebody's database.  The former
is suitable for a pretty picture on the screen, and the latter
is not.  Without resorting to heuristics there is no way to know.

> More generally, I wonder whether fine-grain classification of our
> files might not be better achieved with a reserved keyword in the
> primary header.  The keyword would have value strings coded according
> to some convention.  Such a string could specify that an
> 'application/fits-image' file is an NOAO mosaic, or a DEIMOS mosaic,
> or some other scheme for using a set of IMAGE extensions (plus table
> extensions) to convey some sort of image data.  The same keyword, or
> maybe an analogous keyword, would be used in 'application/fits-table'
> files to say whether the content is a set of tables conveying an event
> list from a high energy experiment, with the schema conforming to some
> convention, or instead the content is a set of tables conveying radio
> aperture synthesis fringe visibilities in conformance with the
> FITS-IDI (Interferometer Data Interchange) convention.

I wholeheartedly endorse such a thing, but I see it as being outside
the scope of the MIME RFC because of the somewhat urgent needs of the
VO community to have some registered MIME types soon.  I would rather
not see the scope of the MIME RFC effort expand to require creation of
new FITS conventions first.

> Usually the client-side implementation of MIME codes is that one
> application code is associated with each code.  It seems to me that it
> should be possible to code a simple program which could decide the
> provenance and probable detailed type of a FITS file by examining the
> headers, even in the absence of a keyword such as the one discussed in
> the previous paragraph, and which could then decide which of a set of
> application programs will be used to process each individual file.
> The classification program would then invoke the proper application.

In the absence of a classification keyword with registered meaning
this is a nasty exercise in heuristics.  It can probably be done
pretty well, but one could equally well say that the "charset"
parameter for MIME types could be guessed by comparing the content of
the file with a sufficiently large database of sample text in various
languages.  I would rather not endorse such a scheme.

In short, I do not see that the fits-image and fits-table subtypes
provide anything useful beyond what is communicated by
application/fits.  The extra bit of classification does not remove the
need to apply a bunch of heuristics in order to figure out what to do
with the file.  It also requires webmasters to engage in a very tricky
classification scheme which is not well supported in the absence of
different file extensions for different MIME types.  I would prefer to
register application/fits and then start exerting effort within the
FITS community to develop some sort of internal keyword/value
registry.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From sla at ucolick.org  Mon Dec  9 16:22:20 2002
From: sla at ucolick.org (Steve Allen)
Date: Mon, 9 Dec 2002 13:22:20 -0800
Subject: [fitsmime] Thoughts on FITS MIME types.
In-Reply-To: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk>
References: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk>
Message-ID: <20021209212220.GA18798@ucolick.org>

On Mon 2002-12-09T17:03:16 +0000, Clive Davenhall hath writ:
>   why does compression using hcompress need its own MIME type (Section
>   4.2).  Should not hcompression also be handled by a content-encoding
>   entry?  Or an I missing something?

I inserted the hcompress entry as a representative for the large range
of MIME types which Bill Joye had found to be in use.  I do not think
it is viable for registration with the IANA, mostly on the grounds that
there is not enough authority behind its usage.

It would be good to see comments from groups who are employing any of
those MIME types, for in the absence of those it is hard to be sure
that the RFC will satisfy their needs.


Note that I did not mention application/fits-group in my previous
post.  I see this type as a gateway to further discussion.  I am
not familiar enough with the nature of its usage to write more,
but I'll pose questions:

Are FITS Random Groups files in sufficiently widespread use that their
users can justify the creation of the one "special" category of
application/fits-group?

If so, would including this type into the RFC pose as a valid means of
pointing the way toward future creation of other specialized MIME
types for FITS?  Or would its inclusion be a bad precedent?

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From dwells at nrao.edu  Mon Dec  9 18:38:12 2002
From: dwells at nrao.edu (Don Wells)
Date: Mon, 9 Dec 2002 18:38:12 -0500
Subject: [fitsmime] Thoughts on FITS MIME types.
In-Reply-To: <20021209212220.GA18798@ucolick.org>
References: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk>
	<20021209212220.GA18798@ucolick.org>
Message-ID: <15861.10596.198024.975244@fits.cv.nrao.edu>

Steve Allen writes:
 > Are FITS Random Groups files in sufficiently widespread use that
 > their users can justify the creation of the one "special" category
 > of application/fits-group?

Bill Cotton tells me that they are still used for interchange between
various synthesis imaging software packages, although usage of the
equivalent BINTABLE constructs (e.g., the FITS-IDI conventions) has
grown steadily during the past decade. 

I recently chose a sample random group file for use in testing
whatever MIME coding scheme we decide to use for such files.  The
sample I chose, after a suggestion by Bill, was the UV data file for
the 'medium-size' problem of the DDT [Dirty Dozen Test] benchmark and
verification package of Classic-AIPS.  The file has a length of about
900_KB.  The random group structure in its PHDU is followed by a
single BINTABLE extension.

 > If so, would including this type into the RFC pose as a valid means
 > of pointing the way toward future creation of other specialized
 > MIME types for FITS?  Or would its inclusion be a bad precedent?

I think that 'application/fits-groups' (or '-group'?) is justified by
the distinctive basic data structure which random groups use.  It is
only fair to warn client-side applications about such files. I am not
talking about the astronomical content of the files, I am talking
about their FITS syntax. The required keywords of the sample file are:

SIMPLE  =                    T /
BITPIX  =                  -32 /
NAXIS   =                    6 /
NAXIS1  =                    0 /No standard image just group
NAXIS2  =                    3 /
NAXIS3  =                    4 /
NAXIS4  =                    1 /
NAXIS5  =                    1 /
NAXIS6  =                    1 /

There is no primary image array, because the product of the dimensions
is zero!  However, this primary header is followed by data records,
rather than by an XTENSION header. In the sample file these binary
data records have a length of 950400 bytes (330r0 FITS records).  They
are followed by a BINTABLE extension.  FITS applications which have
not been programmed to expect random group data records after seeing a
primary header of this type are going to be surprised by this file!
FITS applications which do not check the product of the dimensions
will also be surprised.  So, 'application/fits-groups' is probably
very nearly required in order to properly protect clients, and it
probably will not set a precedent.

				 -=-

Note that CFITSIO supports random groups. My memory is that Bill said
that his CFITSIO implementation transforms the header from random
group syntax to the equivalent BINTABLE syntax, and then uses the
table mechanisms to access the data.  This may mean that all
application programs which use CFITSIO may support random groups in
the same manner that they support BINTABLE extensions. 

-Don
-- 
  Donald C. Wells      Scientist - GBT Project        dwells at nrao.edu
                    http://www.cv.nrao.edu/~dwells
  National Radio Astronomy Observatory                +1-434-296-0277
  520 Edgemont Road,   Charlottesville, Virginia       22903-2475 USA
       (DCW is often in Green Bank, West Virginia, at +1-304-456-2146)


From tam at lheapop.gsfc.nasa.gov  Mon Dec  9 20:28:12 2002
From: tam at lheapop.gsfc.nasa.gov (Tom McGlynn)
Date: Mon, 09 Dec 2002 20:28:12 -0500
Subject: [fitsmime] Thoughts on FITS MIME types.
References: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk>	<20021209212220.GA18798@ucolick.org> <15861.10596.198024.975244@fits.cv.nrao.edu>
Message-ID: <3DF5432C.2070607@lheapop.gsfc.nasa.gov>

I realize that I'm a late comer to this discussion, but I thought I'd 
give some
initial impressions...

It seems to me that this discussion of FITS MIME types is subtly 
misguided.  
I don't think the MIME type
shouldn't be used to distinguish every possible syntax,  it should be 
used to describe
the agreement under which the syntax was created.  E.g., I can use an 
XML MIME type
regardless of the shema....  Similarly I don't have a plethora of 
different HTML mime types
depending upon whether the page includes JavaScript or not.

In a situation where a user is running an application where the type of 
the incoming data
is known, the MIME type is irrelevant -- the user typically ignores
the MIME header.  The problems arise when users need to read FITS data 
of unknown
type.  If this is the case, then the reading program is very likely 
going to be a generic
FITS reader of some kind.  I.e., if I know I'm getting FITS events 
lists, then maybe
I'll be reading the data into the HEASARC's XSELECT program, but if I'm 
linking
to a URL that's a FITS file of unkown type -- so that the MIME types of 
the kind
discussed here wouldn't be known in advance -- then I'm going to putting 
the data
into something much more general like FV that can handle the program 
regardless
of which type it is.  If I really want to use different programs for 
different FITS types
it would be simple enough to build little applications that would know just
enough FITS syntax to start up the correct program.  Such intermediaries 
could
be made infinitely customizable in a way that the MIME types can't
even begin to approximate.

So personally I would advocate a much simpler approach.  I believe that
the Image/FITS specification makes sense for FITS files with no extentions
and with a 2-dimensional image.  These correspond to the notions of the
other image data sets and this very basic notion of FITS  has been used 
outside
of the standard suites of astronomical software (e.g., in versions of 
the XV program).

A second MIME type, application/FITS would be required for all other FITS
data (and would be legal for 2-d images).  Programs which access URLs which
might be FITS data of various types would need to be able to parse the FITS
files -- but I think this is inevitable anyway.   A program which 
understands
a ROSAT events list, might not able to handle one from Integral.  We should
just support a MIME type that tells us that it's legal FITS data coming down
the pipe.

I would think that the IANA would be more agreeable to a request
for just the Image/FITS and Application/FITS MIME types than for
a whole raft of types.  And I really don't think the additional
MIME types gain us much anyway.

    Regards,
    Tom McGlynn

Don Wells wrote:

>Steve Allen writes:
> > Are FITS Random Groups files in sufficiently widespread use that
> > their users can justify the creation of the one "special" category
> > of application/fits-group?
>
>Bill Cotton tells me that they are still used for interchange between
>various synthesis imaging software packages, although usage of the
>equivalent BINTABLE constructs (e.g., the FITS-IDI conventions) has
>grown steadily during the past decade. 
>
>  
>




From sla at ucolick.org  Tue Dec 10 05:26:47 2002
From: sla at ucolick.org (Steve Allen)
Date: Tue, 10 Dec 2002 02:26:47 -0800
Subject: [fitsmime] Thoughts on FITS MIME types.
In-Reply-To: <3DF5432C.2070607@lheapop.gsfc.nasa.gov>
References: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk> <20021209212220.GA18798@ucolick.org> <15861.10596.198024.975244@fits.cv.nrao.edu> <3DF5432C.2070607@lheapop.gsfc.nasa.gov>
Message-ID: <20021210102647.GA32376@ucolick.org>

On Mon 2002-12-09T20:28:12 -0500, Tom McGlynn hath writ:
> the Image/FITS specification makes sense for FITS files with no extentions
> and with a 2-dimensional image.  These correspond to the notions of the
> other image data sets and this very basic notion of FITS  has been used
> outside
> of the standard suites of astronomical software (e.g., in versions of
> the XV program).

The image/* MIME types for AutoCAD encode 3-d vector information as
well as constructive solid geometry (CSG) models (e.g., if we take
these two 2-d shapes, revolve them around their respective axes to
make solids of revolution, and then pivot the results around that
axis, would the resulting pieces of metal collide with each other?).
These AutoCAD formats are much more nearly congruent with the MIME
"model" type than it is with the MIME "image" type.

It is for this reason that I do not believe that image/fits need be
restricted to 2-d images.  Even if it were to be restricted to 2-d, I
believe that WCS table extensions must be legally admitted, for to do
less is to ignore the existence of elements of FITS which are about to
receive the accord of full standard.


Nevertheless, even with recent support from Don Wells for
"application/fits-groups" I remain skeptical about anything other than
"application/fits" because of the precedent set by
"application/postscript".  PostScript has had three different
incarnations (original, Level 2, and Level 3), yet there is only one
MIME type for PostScript.  It is up to the receiving application to
ascertain whether it can handle the incoming PostScript code.

Hopefully the PostScript code contains comments which indicate whether
it is level 3 or level 2.  Ideally it also contains code for
conditional procedures which check the version level of the
interpreter and invoke verbose workaround procedures that enable
an old interpreter to handle new features.  But neither of these is
required.  In their absence, an old interpreter invoked because of
the MIME type "application/postscript" will simply fail.

FITS is already in much better condition than this, for there is no
valid FITS file which should cause HEASARC's fv and similar
applications to fail.


I am hoping to see some input from the VO community here, but at
present my impression remains that the RFC should register
	image/fits
	application/fits
and that the rest of the issues raised here are best solved by
starting up classification efforts within the FITS community.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From lucio at mi.iasf.cnr.it  Tue Dec 10 07:49:41 2002
From: lucio at mi.iasf.cnr.it (Lucio Chiappetti)
Date: Tue, 10 Dec 2002 13:49:41 +0100 (MET)
Subject: [fitsmime] FITS MIME Considerations
Message-ID: <Pine.OSF.4.30.0212101134240.1365-100000@poseidon.mi.iasf.cnr.it>

I apologize if these comments are too naive. Definitely they do not
represent the position of a project, but just of an user (although I had
some part a few years ago in an early AVO prototype
[http://terra.bo.cnr.it/avo/] and we encountered there the issue of
"distributing" FITS (image) files and somehow set it aside because of a
lack of a standard and a SUPPORT to such standard.

Let me ask a preliminary question (which probably many of the participants
to this FITS MIME idea have already answered, but has not been spelled out
explicitly).  Who/what (i.e. which person and which piece of s/w) will
need or benefit of a FITS MIME Content-Type ?

As far as I see, a MIME Content-Type is used mainly by two kind of
applications :

 1) mail clients which receive a data file as a MIME attachment to
    an e-mail message (and I believe this will not be the primary
    usage for FITS ... surelpy we won't encourage people to send large
    FITS files by e-mail cluttering spool areas which are often
    subject to quota or size limitations just to discourage this
    improper usage of e-mail)

 2) web browsers which use the MIME information to call a "viewer"
    this will be the majoritary use

    One could add perhaps a third way ...

 3) dedicated "network viewers" which connect directly via the http
    protocol to a server "giving away" FITS files

In all cases (but particularly in (2) and (3)) this requires the
cooperation of the server in declaring the Content-Type (for an httpd
server based on some AddType directive or mime.types file).

And it requires a suitable arrangement on the client side (a "suitable
arrangement" in most cases could just be "for an unknown Content-Type
offer to Save to Disk"  which is what most plain mail clients and browsers
do anyhow).

To have more useful actions, requires SUPPORT to the (new or unusual, e.g.
FITS) Content-Type from both the server and client sides.

Support from the client side can be arranged on an USER basis, e.g. adding
entries to one's .mailcap file. I'll do it sometimes (e.g. in my pine
e-mail client on Unix I configured to read MS-Word files in plain ASCII
via the "catdoc" utility :-) ).

But the common feeling is that asking to configure one's browser is "too
much" for the generic user. At least that was the feeling in the case of
our early AVO project, and it has still been the case recently in some
other more dedicated contexts.  A colleague comes to me and says "how can
the generic user automatically do this to such a file" and I say "he has
to place this in his own .mailcap" and he says "no, that's too much,
nobody will do it".

Therefore we need some WIDESPREAD SUPPORT at browser level (in case (2)
above ... case (3) it is paradoxically easier, because the average generic
user will probably agree to install a dedicated, pre-canned piece of
software) for some common actions to be taken in case of FITS files (which
cases I'll tell below).

We'd also need some WIDESPREAD SUPPORT at server level. Ideally what one'd
want would be to write an HTML page on one's server containing something
like

  <A HREF=myfile.mytype>This is a FITS file</a>

and having the user at the other end in the browser click on it and have
the appropriate action (view a FITS image, view as text a FITS table, save
to disk a generic FITS file)

So far to achieve only the last action (i.e. force the user to save the
file to disk as binary) I've used a construct via a CGI script :

  <A HREF=/cgi-bin/downloadgw?myfile.mytype>This is a FITS file</a>

with "downloadgw" being a silly script issuing a dummy Content-Type

echo "Content-type: application/x-data"
/bin/echo
/bin/cat $argv

We could have done it adding another non-standard MIME type, but since
there were none registered for FITS we just used the above trick to force
a binary Save to Disk.

What we'd need is to have a mechanism at server level which issues the
appropriate Content-Type header without the need of a CGI.

Note that I'd used "myfile.mytype" above, and not "myfile.fits", because I
consider an useless and annoying limitation the fact to use extension
".fits" (I'd prefer to use ".image", ".spectrum", ".mission_type" or
whatever which makes reference to the INTENDED USAGE).

In principle it won't be that difficult to define rules which tell some
basic kinds of FITS file as it is done in the Unix "magic" files (if we
content with recognising a generic FITS file, it's enough to look for
SIMPLE=T) ... what is unclear to me it's the level of support to "magic"
files by http servers (I see that one of my Apache conf directory contains
a magic file, but have no idea how it uses it). The mechanism is anyhow
superior to the mime.types approach.


What we'd need at the client level is a set of SUPPORTED VIEWERS (maybe
imbedded). In this respect these are the actions I'd expect :

 (a) for a "plain image" to display it, or to present a choice between
     displaying and saving to disk. Definitely we do not want that
     the fact a FITS MIME type causes an automatic display, forbid or
     make difficult to retrieve the file as binary ! Alhough the
     "click vs shift-click" paradigm can be OK.

 (b) for a "plain table" to view it as an HTML formatted table (many
     of us have written such HTML FITS viewers as CGI scripts, but
     it would be handier to have this invoked on the fly)

 (c) for "any other file" to be able to save it to disk.

[a] What do I mean as a "plain image" ?

    a plain image is definitely a NAXIS=2 image, with or without
    extensions required by WCS conventions. I suppose that the
    associated generic viewer should at least render the pixel image. It
    could make use of the WCS if it recognizes it. It should ignore
    extensions which does not recognise, but allow to save the entire
    file on disk.  And possibly should allow a way to display the
    file header.

    a NAXIS=1 image could also be accepted as a plain image (without
    resorting to the idiom I do not like NAXIS=2 NAXIS1=n NAXIS2=1)

    a NAXIS>=3 image could also be accepted, but the default action
    would be to render its first plane (more sophisticated viewers
    could prompt the user for the plane to display)

    a FITS file composed exclusivley by IMAGE extensions could also
    be handled in a way to prompt for the extension to display

    I doubt however whether the latter three cases require each one
    a different MIME type image/fits, application/fits-image1d,
    application/fits-image3d, application/fits-imageset ... or one
    could leave it as "additional non-default actions" to the
    viewer.  After all the viewer will have access to the full FITS
    header !

    I'm not deep enough into MIME to tell a preference between a type
    of image/fits and one of application/fits-image. But probably
    there should be only one of them !

[b] What do I mean by a "plain table" ?

    a plain table is a FITS file composed by a null primary HDU followed
    by a single TABLE extension (although my personal opinion is that
    ASCII tables should be superseded by BINTABLEs they may be still
    in use at catalogue sites).
    The generic viewer should render it either as an HTML formatted
    table, or as the ASCII table records, newline terminated and
    bracketed by <pre></pre>

    a plain table is also a FITS file composed by a null primary HDU
    followed by a single BINTABLE extension.
    Despite of the very different interpretations that the software
    for which the table is intended can assume (e.g. event files,
    response matrices, etc.) it should always make sense to "inspect"
    the table content and present it as an HTML formatted table
    (although some "cells" may be "clickable objects" like e.g. all
    those in a multi-dimensional column).

    There are CGI's which are used to inspect binary tables that
    way. Of course one could always decide that the viewer shows
    only the simplest things, and displays "unsupported" in the
    other cases.

    The table viewer too should allow saving to disk the entire file

    If a file is composed only by a number of BINTABLE extensions,
    a viewer could prompt for the extention to be displayed and
    perform as for a "plain table".

    As for the case [a] above, I'm not deep enough into MIME to tell the
    name for the MIME type. Should one distinguish
    application/fits-table and application/fits-bintable, or just use
    a generic wrapper (application/fits-table or even text/fits !) and
    let the viewer (which is or should be FULLY FITS CAPABLE) to solve
    the difference ?

In conclusion, from the formal point of view I agree with the statement
by William Thompson :

On Fri, 6 Dec 2002, William Thompson wrote:

> MIME types should also be restricted to only those types which have
> been officially recognized by the IAU, i.e. single HDU, random groups,
> and the extensions IMAGE, TABLE, and BINTABLE.  Conventions used by
> specific projects should not be incorporated until they're recognized
> by the IAU.

This means that image/fits-hcompress (whose reason AS DIFFERENT FROM A
compression encoding should be clear to everybody) should wait so far.

For the rest I consider three basic types

  images : image/fits or application/fits-image

  tables : text/fits (!) or application/fits-table and/or
           application/fits-bintable

  generic : application/fits

This does not mean I'm against random groups, only, since I do not know or
use them, I abstain.

However, while I won't see a proliferation beyond this, I have no strong
objections versus a simplified scheme with images on one hand, and any
other generic FITS files on the other.

----------------------------------------------------------------------------
Lucio Chiappetti - IASF/CNR - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.mi.iasf.cnr.it/~lucio/personal.html
----------------------------------------------------------------------------
Berlusconi's contract with Italians - Explained!  (adapted from Usenet 1996)
Contract: verb
1) To shrink or reduce in size - the economy contracted
2) To become infected - he contracted pneumonia when they stopped my welfare
----------------------------------------------------------------------------



From acd at roe.ac.uk  Tue Dec 10 09:43:35 2002
From: acd at roe.ac.uk (Clive Davenhall)
Date: Tue, 10 Dec 2002 14:43:35 +0000 (GMT)
Subject: [fitsmime] Thoughts on FITS MIME types.
In-Reply-To: <20021210102647.GA32376@ucolick.org>
Message-ID: <Pine.OSF.4.10.10212101442560.112919-100000@graemsay.roe.ac.uk>

10/12/02.

Steve Allen wrote:

> I am hoping to see some input from the VO community here, but at
> present my impression remains that the RFC should register
>        image/fits
>        application/fits
> and that the rest of the issues raised here are best solved by
> starting up classification efforts within the FITS community.

I agree.

The proposed classification parameter has much to commend it (and it
could probably be implemented as a FITS keyword, a MIME parameter or, 
indeed, both).  However, it is also potentially a can of worms and working
out a set of permitted values seems likely to be a difficult and protracted
process.  Any standard certainly needs to emerge through the FITS community
and perhaps also through the wider astronomical community (we are really
talking about the classification of astronomical datasets rather than just
FITS files, though of course most astronomical datasets are FITS files).  I
also suspect that there is a small but finite possibility that the effort
would get hopelessly bogged down and nothing useful would emerge.

For these reasons I don't think that we should delay the work on the MIME
types because of the classification parameter, or make the MIME type
depend on the classification parameter.

There is some convergence between a dataset classification scheme and
other work which has been going on in (broadly) the VO area.  Over the
past few years the CDS has developed the UCD (Unified Column Descriptor),
which is a classification scheme for the columns in astronomical tables,
but which might also have some wider applicability.  AstroGrid (the UK
VO Project) is considering developing a so-called astronomical `ontology'
(basically a classification scheme for technical terms) for use in some
aspects of the VO.  These efforts obviously have some commonality with a
classification scheme for datasets, but will take time to come to fruition.

So, to reiterate, a classification scheme for datasets seems a good idea,
but it is a long-term project and the registration of MIME types should
not be delayed because of it.

-----------------------------------------------------------------------------
Clive Davenhall                                      Institute for Astronomy,
e-mail (internet, JANET): acd @ roe.ac.uk        Royal Observatory Edinburgh,
fax from within the UK:   0131-668-8416            Blackford Hill, Edinburgh,
fax from overseas:     +44-131-668-8416                    EH9 3HJ, Scotland.





From sla at ucolick.org  Tue Dec 10 11:57:48 2002
From: sla at ucolick.org (Steve Allen)
Date: Tue, 10 Dec 2002 08:57:48 -0800
Subject: [fitsmime] How it works
In-Reply-To: <Pine.OSF.4.30.0212101134240.1365-100000@poseidon.mi.iasf.cnr.it>
References: <Pine.OSF.4.30.0212101134240.1365-100000@poseidon.mi.iasf.cnr.it>
Message-ID: <20021210165748.GA8594@ucolick.org>

On Tue 2002-12-10T13:49:41 +0100, Lucio Chiappetti hath writ:
> But the common feeling is that asking to configure one's browser is "too
> much" for the generic user. At least that was the feeling in the case of
> our early AVO project, and it has still been the case recently in some
> other more dedicated contexts.  A colleague comes to me and says "how can
> the generic user automatically do this to such a file" and I say "he has
> to place this in his own .mailcap" and he says "no, that's too much,
> nobody will do it".

If most users are using web-based mailreaders, and if the MIME type is
registered, then I strongly suspect the association between image/fits
and an application that can view it will be made by a number of vendors.
Read on to see why I believe this.

> We'd also need some WIDESPREAD SUPPORT at server level. Ideally what one'd
> want would be to write an HTML page on one's server containing something
> like
>
>   <A HREF=myfile.mytype>This is a FITS file</a>
>
> and having the user at the other end in the browser click on it and have
> the appropriate action (view a FITS image, view as text a FITS table, save
> to disk a generic FITS file)

> What we'd need is to have a mechanism at server level which issues the
> appropriate Content-Type header without the need of a CGI.

All webservers are different, but many share a common heritage with
the early NCSA code.  In most webservers there is a configuration file
named mime.types, and this contains lines with each known MIME types
followed by a list of file extensions that are to be classified as that
type.

In current versions of Apache this can be overridden at two levels.
The name of the file may be other than "mime.types" as specified by
the "TypesConfig" directive.  (This is the same mechanism used by many
mail user agents.)  And there may be per-directory overrides for MIME
types that use the "AddType" and "ForceType" directives to associate
any other file extensions with MIME types.

Addressing the "WIDESPREAD SUPPORT" issue, I can vouch that getting a
MIME type registered with the IANA is sufficient to get a MIME type
included into the next release of the "mime.types" file from most
vendors.  I know this because I am already the author of a
registration document on file with the IANA as MIME type "text/vnd.abc".
Shortly after this type was registered it began appearing in the
"mime.types" file for many webservers.

However, the Apache entry for "text/vnd.abc" contains a null list of
file extensions; i.e., ".abc" is not defaulted to be one of our files.
There is no other registered MIME type which asserts ".abc" as the
file extension.  Nevertheless, I believe that the Apache authors
consider abc files to be quite rare, and that they are unwilling to be
seen awarding such a prime piece of namespace to us -- especially when
they have provided a per-directory override that permits lowly webpage
authors to modify the system defaults.  I suspect that ".fits" (and
any other commonly used extensions that are mentioned by the RFC) is
much more likely to be found in future releases of Apache.

Note that the file extension (.fits) indicated by the MIME type
registration is taken to be advisory, not prescriptive.  Nothing
prevents you from using your own file naming scheme, but certain
operating systems provide strong incentives to conform to a small list
of pre-defined extensions.

> For the rest I consider three basic types
>
>   images : image/fits or application/fits-image
>
>   tables : text/fits (!) or application/fits-table and/or
>            application/fits-bintable
>
>   generic : application/fits

It is my impression that after registering MIME types we will find
that the config files which come from the webserver vendors will
associate the ".fits" extension with either "image/fits" or
"application/fits", and that their choice will depend on the wording
of the RFC text.

Even if we were to register only two MIME types this will require an
educational outreach to the FITS community.  Presumably it will be
best to create HOWTO pages that demonstrates the configuration of
various different species of webserver to publish FITS files.  The
HOWTO pages would best be replicated onto all sites that publish
the FITS standard.

> To have more useful actions, requires SUPPORT to the (new or unusual, e.g.
> FITS) Content-Type from both the server and client sides.

If you build it (or register it with the IANA) they will come.

I suspect that once FITS is registered, and once it is understood that
the various VO sites are chock full of astronomical images that
are nice to view, then we will see FITS viewers become much more
commonplace in desktop systems.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From thompson at orpheus.nascom.nasa.gov  Tue Dec 10 15:31:58 2002
From: thompson at orpheus.nascom.nasa.gov (William Thompson)
Date: Tue, 10 Dec 2002 15:31:58 -0500
Subject: [fitsmime] FITS MIME Considerations
References: <Pine.OSF.4.30.0212101134240.1365-100000@poseidon.mi.iasf.cnr.it>
Message-ID: <3DF64F3E.51C96EED@orpheus.nascom.nasa.gov>

Lucio Chiappetti wrote:

	...
 
> Note that I'd used "myfile.mytype" above, and not "myfile.fits", because I
> consider an useless and annoying limitation the fact to use extension
> ".fits" (I'd prefer to use ".image", ".spectrum", ".mission_type" or
> whatever which makes reference to the INTENDED USAGE).

	...

While I'm in general agreement with Lucio's excellently written message, I feel
I must take exception to the above.  The extension of the file should always
tell you what the format of the file is, i.e. whether it is FITS, CDF, netCDF,
HDF, etc., or even one of the image formats, GIF, JPEG, etc.  I feel very
strongly about that.

Not only is this common computer practice, but can be extremely useful.  For
example, one of the resources we use in the SOHO project are the orbit and
attitude files, which are available in several formats, CDF, FITS, and plain
ASCII text.  These files all have the same names, with the appropriate extension
for the file format.  There are also locations where the same image may be
available in FITS and GIF format.  A similar situation exists with documents,
which are also often served in multiple formats, e.g. Word, PDF, PostScript,
etc.

This is slightly off topic, except in that one would presume that the MIME
definition would include references to the typical extensions ".fits", ".fit",
and ".fts".

William Thompson


From William.D.Pence at nasa.gov  Tue Dec 10 18:12:35 2002
From: William.D.Pence at nasa.gov (William Pence)
Date: Tue, 10 Dec 2002 18:12:35 -0500
Subject: [fitsmime] FITS MIME Considerations
References: <Pine.OSF.4.30.0212101134240.1365-100000@poseidon.mi.iasf.cnr.it> <3DF64F3E.51C96EED@orpheus.nascom.nasa.gov>
Message-ID: <3DF674E3.9FEFF3B8@nasa.gov>

William Thompson wrote:
>
> While I'm in general agreement with Lucio's excellently written message, I feel
> I must take exception to the above.  The extension of the file should always
> tell you what the format of the file is, i.e. whether it is FITS, CDF, netCDF,
> HDF, etc., or even one of the image formats, GIF, JPEG, etc.  I feel very
> strongly about that.

Within high-energy astrophysics most missions don't label their data files
with a unique extension like '.fits'.   Since the data are always assumed to
be in FITS format, it would be redundant to add an extra '.fits' to every
filename, not to mention an added burden to users who sometimes have to type
in the long filenames on the command line.   The extension characters are
usually reserved to identify the scientific content of the file, as in these
examples

ad60033000g200270h.evt  (event file)
xh30216010404_b0c.lc    (light curve)

One current mission that does add a FITS suffix to every filename is
XMM-Newton, which has filenames like 
0277_0136550201_M1X00000PTH.FIT
P0136550201R1S004BGSPEC1001.FTZ

where .FTZ means it is a gzip compressed FITS file.

Given this widespread practice of not using a standard suffix for every FITS
format file, I don't think we can expect to introduce such a standard now. 
Personally, I think the flexibility and freedom in specifying FITS filenames
is much more important to users than the advantages that would be gained by
always using the same .fits suffix (such as making it easier to assign mime
types to the files).  

Bill Pence
-- 
____________________________________________________________________
Dr. William Pence                          William.D.Pence at nasa.gov
NASA/GSFC Code 662         HEASARC         +1-301-286-4599 (voice)     
Greenbelt MD 20771                         +1-301-286-1684 (fax)


From lucio at mi.iasf.cnr.it  Wed Dec 11 12:47:31 2002
From: lucio at mi.iasf.cnr.it (Lucio Chiappetti)
Date: Wed, 11 Dec 2002 18:47:31 +0100 (MET)
Subject: [fitsmime] FITS MIME Considerations
In-Reply-To: <3DF674E3.9FEFF3B8@nasa.gov>
Message-ID: <Pine.OSF.4.30.0212111815150.1365-100000@poseidon.mi.iasf.cnr.it>

On Tue, 10 Dec 2002, William Pence wrote:

> Since the data are always assumed to be in FITS format, it would be
> redundant to add an extra '.fits' to every filename,

> ad60033000g200270h.evt  (event file)
> xh30216010404_b0c.lc    (light curve)

I agree with Bill (I actually expected him to quote also .rmf (response
matrix files), .arf (ancillary response files) and .pha (spectra) !)

> One current mission that does add a FITS suffix to every filename is
> XMM-Newton,

unfortunately ! But their s/w has been designed more by engineers than
by astronomers :-)

On Tue, 10 Dec 2002, William Thompson wrote:

> The extension of the file should always tell you what the format of
> the file is, i.e. whether it is FITS, CDF, netC$ HDF, etc., or even
> one of the image formats, GIF, JPEG, etc.  I feel very strongly about
> that.

My feeling is opposite. If I go into a library in an English-speaking
country, I do not care to find shelfs labelled as "books in english", but
more by topic, history, fiction, science and so on. So I'd like that the
extension tells me (human user) that the file is a spectrum, an image, a
response matrix.

> This is slightly off topic, except in that one would presume that the
> MIME definition would include references to the typical extensions
> ".fits", ".fi$ and ".fts".

exactly, the only reason this argument is on topic here are those
references ! See below comments on Steve Allen's statements.


On Tue, 10 Dec 2002, Steve Allen wrote:

> All webservers are different, but many share a common heritage with
> the early NCSA code.  In most webservers there is a configuration file
> named mime.types, and this contains lines with each known MIME types
> followed by a list of file extensions that are to be classified as that
> type.

I should know, I still use an NCSA server for my own little private httpd.
Although I did not dare to alter mime.types !

> In current versions of Apache this can be overridden at two levels.
> [...] And there may be per-directory overrides for MIME
> types that use the "AddType" and "ForceType" directives to associate
> any other file extensions with MIME types.

I did use AddType to add features (about CGI's and to treat .html as
.shtml) but again I did not dare to add non-standard types.

I believe there is a third way in Apache ... I just checked in the conf
directory of our new official server and noticed a file called "magic"
(like the /etc/magic of Unix). So I browsed into the Apache documentation
and found it has a module called mod_mime_magic. So it looks like in
principle an httpd server could be configured to use a magic file to tell
what a file is. I also found other things I never bothered to look after,
like "output filters".

After all, there will be many users and browsers and fewer VO sites and
httpd servers, so it is reasonable to suppose that the server
administrators will be more knowledgeable and willing (then the users) to
make custom configurations, and decide that ".xyz" files are FITS images,
".uvw" are tables, and so on.

> Note that the file extension (.fits) indicated by the MIME type
> registration is taken to be advisory, not prescriptive.  Nothing

You mean what is in the RFC is advisory, don't you ? But once it makes
its way to the mime.types or mailcap becomes prescriptive for the server
or the browser !

I believe we should be careful. I presume that if a server (via mime.types
or magic) can identify a ".xyz" file and issue an http header with a
Content-Type of application/fits-whatever, the browser will obey the
Content_Type irrespective of the file extension announced. Is this true ?

We should allow the (few) server administrators the maximum freedom to
associate files with MIME types (specially if there will be many for FITS
variations). At the same time we should prevent the default mailcap of the
(greater number of ) browsers to override the choice of the
administrators.

> certain operating systems provide strong incentives to conform to a
> small list of pre-defined extensions.

That's exactly the nuisance we should avoid. Recently we had to use some
Windows/NT PCs to control the mask cutting machine for the VIMOS
spectrograph, Such PC exchanges "orders" and "reports" with the rest of
the ESO (Unix) system, so we had to devise a file extension naming, and we
found naturally to choose for our files (all ASCII) extensions like mmj,
mij, mdj for the "job orders" and mmt, mir, mdr for the "termination
reports". But for NT some of those types are by default associated to some
other cllickable action ...

> It is my impression that after registering MIME types we will find
> that the config files which come from the webserver vendors will
> associate the ".fits" extension with either "image/fits" or
> "application/fits", and that their choice will depend on the wording
> of the RFC text.

That's why we should be careful. Why should one choose one or the other ?
May be our wording should be as neutral as possible. Or, as you say

> Even if we were to register only two MIME types this will require an
> educational outreach to the FITS community.

----------------------------------------------------------------------------
Lucio Chiappetti - IASF/CNR - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.mi.iasf.cnr.it/~lucio/personal.html
----------------------------------------------------------------------------
As of 10 Dec 2002 the Rectors of all Italian Universities resigned to
protest against the cuts to University and Research funds in the 2003
Budget Law.
----------------------------------------------------------------------------



From sla at ucolick.org  Wed Dec 11 14:45:31 2002
From: sla at ucolick.org (Steve Allen)
Date: Wed, 11 Dec 2002 11:45:31 -0800
Subject: [fitsmime] FITS MIME Considerations
In-Reply-To: <Pine.OSF.4.30.0212111815150.1365-100000@poseidon.mi.iasf.cnr.it>
References: <3DF674E3.9FEFF3B8@nasa.gov> <Pine.OSF.4.30.0212111815150.1365-100000@poseidon.mi.iasf.cnr.it>
Message-ID: <20021211194531.GA7781@ucolick.org>

On Wed 2002-12-11T18:47:31 +0100, Lucio Chiappetti hath writ:
> > Note that the file extension (.fits) indicated by the MIME type
> > registration is taken to be advisory, not prescriptive.  Nothing
>
> You mean what is in the RFC is advisory, don't you ? But once it makes
> its way to the mime.types or mailcap becomes prescriptive for the server
> or the browser !

Sorry, I was being very precise.  Technically the RFC is required
because the MIME types are top-level, but aside from that the only
purpose it serves is as a wrapper.  The important contents are the
IANA registrations for the individual media types.

> I believe we should be careful. I presume that if a server (via mime.types
> or magic) can identify a ".xyz" file and issue an http header with a
> Content-Type of application/fits-whatever, the browser will obey the
> Content_Type irrespective of the file extension announced. Is this true ?

Yes, this is prescribed in RFC 2616 (HTTP 1.1), section 7.2.1.

> We should allow the (few) server administrators the maximum freedom to
> associate files with MIME types (specially if there will be many for FITS
> variations). At the same time we should prevent the default mailcap of the
> (greater number of ) browsers to override the choice of the
> administrators.

It is my impression that the media type registrations definitely
should claim the namespace for the file extension ".fits", but that
they should also include disclaiming text.  The beauty of the
IETF/IANA processes is that they are nearly free form and admit a lot
of commentary.  Perhaps something more like this:

   File extension(s): fits

   This file extension should not be interpreted as a prescription.

   The FITS standard originated in the era when files were stored and
   exchanged via magnetic tape; it does not prescribe any nomenclature
   for files on disk.
   Various sites within the FITS community have long-established
   practices where files are presumed to be FITS by context.
   File extensions used at such sites commonly indicate content
   of the file instead of the data format.

   In the absence of other information it is reasonably safe to
   presume that a file name ending in ".fits" is intended to be a FITS
   file.  Nevertheless, there are other commonly used extensions;
   e.g., ".fit", ".fts", and many others not suitable for listing in a
   media type registration.

Getting back to the timing of the effort, I am of the impression that
I should attempt to publish a new draft of the RFC, which incorporates
input from up-to-date discussions, early in the year 2003.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From thompson at orpheus.nascom.nasa.gov  Wed Dec 11 18:29:02 2002
From: thompson at orpheus.nascom.nasa.gov (William Thompson)
Date: Wed, 11 Dec 2002 18:29:02 -0500
Subject: [fitsmime] FITS MIME Considerations
Message-ID: <3DF7CA3E.2F314125@orpheus.nascom.nasa.gov>

Lucio Chiappetti wrote:
> 
> On Tue, 10 Dec 2002, William Pence wrote:
> 
> > Since the data are always assumed to be in FITS format, it would be
> > redundant to add an extra '.fits' to every filename,
> 
> > ad60033000g200270h.evt  (event file)
> > xh30216010404_b0c.lc    (light curve)
> 
> I agree with Bill (I actually expected him to quote also .rmf (response
> matrix files), .arf (ancillary response files) and .pha (spectra) !)
> 
> > One current mission that does add a FITS suffix to every filename is
> > XMM-Newton,
> 
> unfortunately ! But their s/w has been designed more by engineers than
> by astronomers :-)
> 
> On Tue, 10 Dec 2002, William Thompson wrote:
> 
> > The extension of the file should always tell you what the format of
> > the file is, i.e. whether it is FITS, CDF, netC$ HDF, etc., or even
> > one of the image formats, GIF, JPEG, etc.  I feel very strongly about
> > that.
> 
> My feeling is opposite. If I go into a library in an English-speaking
> country, I do not care to find shelfs labelled as "books in english", but
> more by topic, history, fiction, science and so on. So I'd like that the
> extension tells me (human user) that the file is a spectrum, an image, a
> response matrix.

I guess this just reflects the different worlds we live in.  For a number of
people on this list, it seems that the answer is "of course it's FITS, what else
would it be?"  In my world, it's not as obvious as that.  For the SOHO project
there was debate whether we should go with FITS or CDF, and we ended up using
both, depending on the instrument.  They're both considered standards for
portions of our field, and I've certainly seen data in other formats as well.

To use your analogy, I can't go into the bookstore and just assume that all the
books are going to be in English.  Finding the same book translated into
multiple languages is also a strong possibility, and I expect the book cover to
tell me which one I'm going to be able to read.

Of course having the filename tell you whether the file contains an image or a
spectrum is extremely useful, but telling you what format the data is in is
fundamental.

William Thompson


From William.D.Pence at nasa.gov  Fri Dec 13 16:40:30 2002
From: William.D.Pence at nasa.gov (William Pence)
Date: Fri, 13 Dec 2002 16:40:30 -0500
Subject: [fitsmime] Thoughts on FITS MIME types.
References: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk> <20021209212220.GA18798@ucolick.org> <15861.10596.198024.975244@fits.cv.nrao.edu> <3DF5432C.2070607@lheapop.gsfc.nasa.gov> <20021210102647.GA32376@ucolick.org>
Message-ID: <3DFA53CE.4B5B3891@nasa.gov>

Steve Allen wrote:
> 
> I am hoping to see some input from the VO community here, but at
> present my impression remains that the RFC should register
>         image/fits
>         application/fits
> and that the rest of the issues raised here are best solved by
> starting up classification efforts within the FITS community.

To add my $0.02 to this discussion, I would agree with Steve.  Also, I tend
to think image/fits should only be applied to 2-dimensional images,  since
there is a big step up in complexity (possibly requiring user interactions),
to deal with display of 3-D data cubes.

Regarding Random Groups, I currently don't see a strong need for a special
application/fits-group mimetype for that type of file.  I would think that
any application that could be associated with 'application/fits' should be
expected to not choke on a random groups FITS files.  The application would
not necessarily have to do anything intelligent with the data records, but
it should be able to at least deal with the primary header keywords, and be
able to move to any following extensions correctly.  This is all that fv is
able to do, and it is not hard to implement.  Basically, the application
just has to look for 2 keywords in the primary header:

NAXIS1 = 0
GROUPS = T

If both these are present, then the application should procede as if NAXIS1
= 1.  This then allows it to correctly compute the size of the random group
records, and move to any following extensions.  (With hindsight, it seems to
me it would have been much better to simply use the presence of the keyword

GROUPS = T

to denote the presence of random groups, instead of inventing the awkward
NAXIS1 = 0 business, but we're stuck with it now).

Bill
-- 
____________________________________________________________________
Dr. William Pence                          William.D.Pence at nasa.gov
NASA/GSFC Code 662         HEASARC         +1-301-286-4599 (voice)     
Greenbelt MD 20771                         +1-301-286-1684 (fax)


From sla at ucolick.org  Fri Dec 13 17:43:08 2002
From: sla at ucolick.org (Steve Allen)
Date: Fri, 13 Dec 2002 14:43:08 -0800
Subject: [fitsmime] Thoughts on FITS MIME types.
In-Reply-To: <3DFA53CE.4B5B3891@nasa.gov>
References: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk> <20021209212220.GA18798@ucolick.org> <15861.10596.198024.975244@fits.cv.nrao.edu> <3DF5432C.2070607@lheapop.gsfc.nasa.gov> <20021210102647.GA32376@ucolick.org> <3DFA53CE.4B5B3891@nasa.gov>
Message-ID: <20021213224308.GA3006@ucolick.org>

On Fri 2002-12-13T16:40:30 -0500, William Pence hath writ:
> to think image/fits should only be applied to 2-dimensional images,  since
> there is a big step up in complexity (possibly requiring user interactions),
> to deal with display of 3-D data cubes.

But what about

NAXIS   = 4
NAXIS1  = 1024
NAXIS2  = 1024
NAXIS3  = 1
NAXIS4  = 1

?

This is the same bone of confusion that brought WCSAXES into the WCS
papers.

This is really a 2-d image, and to be unable to display it is to
deprive a FITS audience from viewing many files that conform to
suggestions in the original FITS paper.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93


From William.D.Pence at nasa.gov  Fri Dec 13 17:59:40 2002
From: William.D.Pence at nasa.gov (William Pence)
Date: Fri, 13 Dec 2002 17:59:40 -0500
Subject: [fitsmime] Thoughts on FITS MIME types.
References: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk> <20021209212220.GA18798@ucolick.org> <15861.10596.198024.975244@fits.cv.nrao.edu> <3DF5432C.2070607@lheapop.gsfc.nasa.gov> <20021210102647.GA32376@ucolick.org> <3DFA53CE.4B5B3891@nasa.gov> <20021213224308.GA3006@ucolick.org>
Message-ID: <3DFA665C.E33411CF@nasa.gov>

Steve Allen wrote:
> 
> On Fri 2002-12-13T16:40:30 -0500, William Pence hath writ:
> > to think image/fits should only be applied to 2-dimensional images,  since
> > there is a big step up in complexity (possibly requiring user interactions),
> > to deal with display of 3-D data cubes.
> 
> But what about
> 
> NAXIS   = 4
> NAXIS1  = 1024
> NAXIS2  = 1024
> NAXIS3  = 1
> NAXIS4  = 1
> 
> ?
> 
> This is the same bone of confusion that brought WCSAXES into the WCS
> papers.
> 
> This is really a 2-d image, and to be unable to display it is to
> deprive a FITS audience from viewing many files that conform to
> suggestions in the original FITS paper.

Yes, I would agree that this is really a 2-d image and could be
appropriately given a mime type of image/fits.

Bill


From sla at ucolick.org  Sat Dec 14 12:21:02 2002
From: sla at ucolick.org (Steve Allen)
Date: Sat, 14 Dec 2002 09:21:02 -0800
Subject: [fitsmime] image/* commonly refers to 3 dimensions
In-Reply-To: <3DFA665C.E33411CF@nasa.gov>
References: <Pine.OSF.4.10.10212091702410.106386-100000@graemsay.roe.ac.uk> <20021209212220.GA18798@ucolick.org> <15861.10596.198024.975244@fits.cv.nrao.edu> <3DF5432C.2070607@lheapop.gsfc.nasa.gov> <20021210102647.GA32376@ucolick.org> <3DFA53CE.4B5B3891@nasa.gov> <20021213224308.GA3006@ucolick.org> <3DFA665C.E33411CF@nasa.gov>
Message-ID: <20021214172102.GA26909@ucolick.org>

On Fri 2002-12-13T17:59:40 -0500, William Pence hath writ:
[regarding a 4-d FITS array with 2 degenerate axes]
> Yes, I would agree that this is really a 2-d image and could be
> appropriately given a mime type of image/fits.

In support of the notion that any dimension of array in a PHDU should
qualify as image/fits, I point out that image/gif includes
3-dimensional images with which every web user is annoyingly familiar.

Animated GIFs.

--
Steve Allen          UCO/Lick Observatory       Santa Cruz, CA 95064
sla at ucolick.org      Voice: +1 831 459 3046     http://www.ucolick.org/~sla
PGP: 1024/E46978C5   F6 78 D1 10 62 94 8F 2E    49 89 0E FE 26 B4 14 93