[fitswcs] Draft#3 of FITS BoF report for review

[Don Wells]

Dear FITS WCS ad-hoc-TG,

I append a third draft of the report which I propose to post to
sci.astro.fits/fitsbits *tomorrow*, Friday the 13th, after I have applied
corrections/additions which any of you may suggest. My goal is that
this report should reflect the consensus of this TG (the fitswcs
subscribers) and should summarize our work over the past six
months. In particular, if you object to any details of my discussion,
please inform me and/or [fitswcs]. 

Corrections/additions to draft#1 were suggested to me by Doug Mink,
Pat Wallace and Eric Greisen.  Corrections/additions to draft#2 were
suggested to me by Arnold Rots, Doug Mink, Doug Tody and Jonathan
mcDowell. I append the diff against draft#2.

-Don

			   -=- DRAFT#3 -=-

	   ADASS-FITS-BoF annual report (WCS negotiations)

	      Don Wells [Chair, IAU FITS Working Group]

			      1998-11-12

In recent years the FITS BoF [Birds-of-a-Feather] sessions at ADASS
meetings have been the primary annual face-to-face meeting for the
FITS community, and the agenda of each BoF has acted as an annual
report for FITS. I have prepared this memo as a written version of the
three viewgraphs which I used at the ADASS'98 FITS BoF, held at
Univ. of Illinois late in the afternoon of Monday November 2nd. There
were 30-40 people present in the room.

1. The NASA NOST FITS Standard

A technical panel under the leadership of Bob Hanisch has been working
on this codification of all FITS Agreements through the end of 1997. A
draft version was submitted to the community for review in the spring
of 1998, and 140 comments were received from the worldwide FITS
community. The technical panel has considered each of these comments,
and has decided on responses to them. Bob Hanisch was not able to
attend ADASS'98, so the secretary of the panel, Bill Pence, read
Hanisch's report to the BoF. The panel expects to send out the
responses soon, and to have a new version of the Standard ready for
review within a few months. 

The ultimate goal of this effort is to produce a definitive version of
the FITS Standard, a version which will be approved by the three FITS
regional committees and by the IAU FITS Working Group, so that it can
replace the FITS papers.  Don Wells stated that he would like for this
definitive version of the FITS Standard to be published in A&A
Supplement.

2. The FITS Y2K (DATE-OBS) Agreement

This agreement was adopted by the IAU-FWG in November 1997, one year
ago. Don Wells pointed out that full interchange of the new notation
is to begin 1999-01-01, only two months from now. He asked who had
changed code in response to this agreement, and roughly half of those
present raised their hands.

3. IAU Comm5 Task Group [TG] - Designations and OBJECT strings

Don Wells reminded the BoF that the IAU TG on Designations has asked
that the maximum lengths of OBJECT strings in interchange be increased
to at least 26 characters in order to support the IAU standard object
designation syntax. The NOST technical panel has addressed this issue
and has decided that all string values in FITS headers may be up to 68
characters in length, which will solve the technical aspect of this
problem. The real goal is to use object designations which will not be
ambiguous when they appear in archival databases; in particular,
comments about filter choices, weather, etc., should not be appended
to object names in OBJECT strings. Ideally the IAU prefixes and
coordinate syntax will be implemented and used in data acquisition
systems. Wells urged the BoF to 'help stop namespace pollution!'

4. WCS [World Coordinate System] negotiations

Don Wells reminded the BoF that there were a number of WCS issues
outstanding after the ADASS'97 [Sonthofen] BoF, even though
significant progress was made in negotiations during ADASS'97. Some of
the differences of opinion had prevented conclusion of negotiations
for several years. Wells reported that in March 1998 he had created a
small ad hoc task group [TG] to work on the outstanding technical
issues. This TG was expanded to about 15 people in July 1998, and much
progress was made, with the result that Wells reported to the BoF that
a working consensus now exists. The final major steps of the
negotiations occurred during the 24 hours preceding the ADASS'98 BoF.
The WCS Agreement which the TG is producing will be the most complex
FITS agreement to date; we have been negotiating it for more than a
decade.  The items below are not the complete agreement, but only the
set of important features which were discussed at the BoF.

a) Split G&C into two papers

The TG will recommend that the Greisen & Calabretta draft WCS paper be
split into two papers. The first paper will have a title something
like 'Generalized Representations of World Coordinate Systems in
FITS', and will specify meta-rules for FITS WCS conventions. The
second paper will specify the WCS conventions for celestial
coordinates. The meta-rules are needed because the TG expects that
eventually there will be a third paper for the spectroscopic case and
maybe another paper on time axes.

b) Linear transformation to use CDi_j keywords

The TG will recommend that the IRAF CDi_j rotation matrix keywords be
adopted instead of the PCiiijjj keywords which were suggested in
previous drafts of the Greisen & Calabretta paper.  This means that
CDELTi and CROTAi will not be used. It also means that the WCS
agreement will have backwards compatibility with several existing data
archives.

c) Multiple-WCS support

The TG will recommend that optional additional sets of WCS keywords in
FITS headers be distinguished from the default set of WCS keywords and
from each other by appending a single alphabetic character [A-Z] to
all of the WCS keywords. Examples of such keywords could be CRPIX1B,
CD2_1C, CRVAL3D and CTYPE2A. Note that the default set of WCS keywords
in these cases would still be CRPIX1, CD2_1, CRVAL3 and CTYPE2.
Datasystems which do not support the multiple WCS notation will need
to regard WCS keywords with trailing non-numeric characters as
unrecognized keywords. Discussion of this idea during the BoF
exposed new examples of usage which added to the cases which have
convinced the TG that multiple-WCS capability will be valuable.

The optional appended version code character will restrict the basic
WCS keywords to seven characters. Therefore, the TG will recommend
that WCS axis numbers be in the range 1-99 (i.e., axes 100-999 will
not be supported).

d) New distorted-projection capability to be added

The TG will recommend that a set of distortion terms analogous to
those used in the DSS [Digital Sky Survey] be added to the basic TAN
and ARC projections in order to support a variety of optical imaging
systems which produce distorted geometries without requiring
re-gridding of the vast quantities of data being produced. The goal is
to agree on a set of terms which will represent the geometric
distortions of all existing optical cameras.

The set will include radial terms, analogous to those in the 'ZPN'
projection described in the Greisen & Calabretta draft. The majority
of imaging systems will be supported by these radial terms. (The
'radial' terms used in the DSS are X(X^2+Y^2), Y(X^2+Y^2), X(X^2+Y^2)^2
and Y(X^2+Y^2)^2; these implement an elliptically-symmetric radial
distortion.)

The non-radial terms, such as XY^2 and X^2Y, will be particularly
useful for re-imaging cameras like HST's WFPC-1 and WFPC-2.  Several
TG members believe that such cameras have two 'center-points' for
their axes, the tangent point of the main telescope and the centers of
the field-flattening lenses of the individual cameras; probably two
keywords will be defined to specify a center point offset for the
polynomial terms (the main telescope's TAN geometry center point will
be specified by CRPIXi). Several TG members intend to collect
information about existing re-imaging cameras to verify this
conjecture.

Work is going on to formulate models that describe the small
systematic distortions seen in Schmidt astrometry, relative to the ARC
projection, and this important special case may justify explicit
support in the FITS standard.  Whether or not the TG recommends that
terms for such models be added to the set of distortion terms, some
version of the pixel correction images described in Appendix A of the
Greisen & Calabretta draft will probably still be specified to
describe the residual random distortion field for the highest
precision, widest field applications. The TG expects that most optical
cameras will be represented satisfactorily by the polynomial terms
alone.

The TG will recommend that all WCS parameter values be conveyed by
keywords PVi_k (PVk_i?), where i is the index of the parameter and k is the
index of the axis. This notation will replace the PROJPi keywords
suggested in the Greisen & Calabretta draft.

The axis index for WCS keywords will be restricted to 99 or less; note
that if the axis index k is 9 or less (the usual case), the parameter
index i in PVi_k may range up to 999 without exceeding the seven
character limit set by the plan to use an optional appended character
for multiple-WCS support.

e) WCS discussions and  decisions during the BoF session

Don Wells encouraged discussion of items (a), (b), (c) and (d). In the
course of the discussion, it became clear that items (a), (b) and (c)
had firm support from those present at the BoF. The following issues
were discussed:

- Wells asked whether the distortion projection should be given some
  new name to distinguish it from TAN and/or ARC; some TG members have
  worried that the complicated polynomial would confuse newcomers to
  FITS WCS who need to know that the TAN projection is appropriate for
  most optical imagery. It became clear that there was a consensus
  favoring adding the new terms to both the TAN and the ARC
  projections rather than creating a new projection. Although the
  leading radial term of the polynomial is sufficient to represent the
  difference between TAN and ARC (the DSS uses TAN rather than ARC, as
  you would expect for a Schmidt camera!), maintaining the distinction
  has a tutorial advantage.

- A number of people at the BoF argued that the order of the indices
  in the PV keyword should be swapped, to PVk_i; presumably the TG
  will consider this modification.

- Wells raised the issue of whether all parameters of a projection
  should be restricted to the 'Dec-like' axis (as it was with CROTAk),
  or alternatively should be associated in some agreed-upon fashion
  with both axes of a celestial projection. Some TG members have
  preferred to retain the former convention, but there was a clear
  consensus at the BoF that parameters should be associated with both
  axes. This convention will have the advantage that analogous
  polynomial terms associated with the two axes will have the same
  indices (one of the drafts considered by the TG had the indices of
  the second axis offset by 20).

- Doug Tody suggested that axis_k=0 be used for specifing general
  parameters that apply to the entire WCS of a multi-WCS group; the TG
  will need to consider the implications of this detail of the rules.

- Tody also raised the issue of whether the datatype of a PV keyword
  should be restricted to numerical, or defined by the WCS function
  type to which the parameter is assigned (this case is unusual in
  that the PV keywords are not fully defined, rather their usage is
  defined by each WCS).

- Some people at the BoF regretted the demise of CROTAi and,
  especially, CDELTi, since they are so easy to use and feature in
  many existing files. They are suggesting that CDELTi and CROTAi
  could function as functional synonyms of CDi_j. (WCS reading code
  will have to support CDELTi and CROTAi forever because there are
  many existing files that use these keywords.  The question is
  whether they are to be "deprecated" or will be recognized as
  "official synonyms".)

Upon a motion by Doug Mink, an informal vote was taken to express
general support for the work of the TG as described above, and the
show of hands in support was unanimous. At this point in the session
Don Wells told Eric Greisen that he and Mark Calabretta now had the
working consensus which they had specified as a precondition before
undertaking the next round of revision of the WCS draft paper.

f) WCStools paper in ADASS'98 session

Paper T6.2 ('WCSTools: an Image Astrometry Toolkit') was presented by
Doug Mink in the afternoon session on Tuesday, the day after the FITS
BoF session. The final sentence of the abstract is: 'The proposed FITS
WCS standard is being tracked, and interim formats are being
supported.' During the questions after Doug Mink spoke, Don Wells
exhorted the community to include proper WCS notations in the headers
of all optical imagery. Mink's WCSTools software is available at:
http://tdc-www.harvard.edu/software/wcstools/.

g) New members for the WCS TG?

Don Wells told the BoF that he is prepared to add new members to the
ad hoc WCS task group, but that such people should understand that
they are expected to contribute to the solutions to FITS WCS
problems. Wells is especially interested in adding people with
knowledge of the geometry of re-imaging cameras, or of Schmidt camera
geometry or general knowledge of precision astrometry, in order to
complete the design of the distortion correction terms for the TAN and
ARC projections.

5. Another Issue discussed at the BoF

One person (the Chair does not recall who it was) reported having
encountered a FITS tape which was not blocked in accordance with the
FITS Blocking Agreement (Section 4.6 in the NOST 100-1.2 [1998-04-02]
version of the FITS Standards). The people present at the BoF deplored
this, of course.

 [NOTE: the text of this report has been reviewed by the WCS TG, and
    corrections/additions suggested by TG members have been made.]

1c1
< 			   -=- DRAFT#3 -=-
---
> 			   -=- DRAFT#2 -=-
7c7
< 			      1998-11-12
---
> 			      1998-11-10
45c45
< 3. IAU Comm5 Task Group [TG] - Designations and OBJECT strings
---
> 3. IAU Comm5 TG-Designations OBJECT strings
100c100
< FITS headers be distinguished from the default set of WCS keywords and
---
> FITS headers be distinguished from the active set of WCS keywords and
103c103
< CD2_1C, CRVAL3D and CTYPE2A. Note that the default set of WCS keywords
---
> CD2_1C, CRVAL3D and CTYPE2A. Note that the active set of WCS keywords
107c107
< unrecognized keywords. Discussion of this idea during the BoF
---
> unrecognized comment keywords. Discussion of this idea during the BoF
157c157
< keywords PVi_k (PVk_i?), where i is the index of the parameter and k is the
---
> keywords PVi_k, where i is the index of the parameter and k is the
167c167
< e) WCS discussions and  decisions during the BoF session
---
> e) WCS decisions during the BoF session
171,220c171,194
< had firm support from those present at the BoF. The following issues
< were discussed:
< 
< - Wells asked whether the distortion projection should be given some
<   new name to distinguish it from TAN and/or ARC; some TG members have
<   worried that the complicated polynomial would confuse newcomers to
<   FITS WCS who need to know that the TAN projection is appropriate for
<   most optical imagery. It became clear that there was a consensus
<   favoring adding the new terms to both the TAN and the ARC
<   projections rather than creating a new projection. Although the
<   leading radial term of the polynomial is sufficient to represent the
<   difference between TAN and ARC (the DSS uses TAN rather than ARC, as
<   you would expect for a Schmidt camera!), maintaining the distinction
<   has a tutorial advantage.
< 
< - A number of people at the BoF argued that the order of the indices
<   in the PV keyword should be swapped, to PVk_i; presumably the TG
<   will consider this modification.
< 
< - Wells raised the issue of whether all parameters of a projection
<   should be restricted to the 'Dec-like' axis (as it was with CROTAk),
<   or alternatively should be associated in some agreed-upon fashion
<   with both axes of a celestial projection. Some TG members have
<   preferred to retain the former convention, but there was a clear
<   consensus at the BoF that parameters should be associated with both
<   axes. This convention will have the advantage that analogous
<   polynomial terms associated with the two axes will have the same
<   indices (one of the drafts considered by the TG had the indices of
<   the second axis offset by 20).
< 
< - Doug Tody suggested that axis_k=0 be used for specifing general
<   parameters that apply to the entire WCS of a multi-WCS group; the TG
<   will need to consider the implications of this detail of the rules.
< 
< - Tody also raised the issue of whether the datatype of a PV keyword
<   should be restricted to numerical, or defined by the WCS function
<   type to which the parameter is assigned (this case is unusual in
<   that the PV keywords are not fully defined, rather their usage is
<   defined by each WCS).
< 
< - Some people at the BoF regretted the demise of CROTAi and,
<   especially, CDELTi, since they are so easy to use and feature in
<   many existing files. They are suggesting that CDELTi and CROTAi
<   could function as functional synonyms of CDi_j. (WCS reading code
<   will have to support CDELTi and CROTAi forever because there are
<   many existing files that use these keywords.  The question is
<   whether they are to be "deprecated" or will be recognized as
<   "official synonyms".)
< 
< Upon a motion by Doug Mink, an informal vote was taken to express
---
> had firm support from those present at the BoF. Wells asked whether
> the distortion projection should be given some new name to distinguish
> it from TAN and/or ARC; some TG members have worried that the
> complicated polynomial would confuse newcomers to FITS WCS who need to
> know that the TAN projection is appropriate for most optical
> imagery. It became clear that there was a consensus favoring adding
> the new terms to both the TAN and the ARC projections rather than
> creating a new projection. Although the leading radial term of the
> polynomial is sufficient to represent the difference between TAN and
> ARC (the DSS uses TAN rather than ARC, as you would expect for a
> Schmidt camera!), maintaining the distinction has a tutorial
> advantage. Several people at the BoF argued that the order of the
> indices in the PV keyword should be swapped, to PVk_i; presumably the
> TG will consider this modification. Wells raised the issue of whether
> all parameters of a projection should be restricted to the 'Dec-like'
> axis (as it was with CROTAk), or alternatively should be
> associated in some agreed-upon fashion with both axes
> of a celestial projection. Some TG members have preferred to retain
> the former convention, but there was a clear consensus at the BoF that
> parameters should be associated with both axes. This convention
> will have the advantage that analogous polynomial terms associated
> with the two axes will have the same indices (one of the drafts
> considered by the TG had the indices of the second axis offset by
> 20). Upon a motion by Doug Mink, an informal vote was taken to express
248,255d221
< 
< 5. Another Issue discussed at the BoF
< 
< One person (the Chair does not recall who it was) reported having
< encountered a FITS tape which was not blocked in accordance with the
< FITS Blocking Agreement (Section 4.6 in the NOST 100-1.2 [1998-04-02]
< version of the FITS Standards). The people present at the BoF deplored
< this, of course.