[fitswcs] Paper III progress
Eric Greisen
egreisen at nrao.edu
Thu Sep 1 19:02:55 EDT 2005
I have re-worked Paper III some. Sections in blue are relatively
recent changes made before the regional committees voted. All
wordings in red have been made since, mostly after the paper was
submitted to Astroph and A&A.
I have yet to hear from A&A's referee. I have held on to this
response to add responses to the referee, but I think I should release
this version to all the people who spent such time reviewing this
document so far. See
http://www.aoc.nrao.edu/~egreisen/scs_050829.ps.gz
http://www.aoc.nrao.edu/~egreisen/scs_050829.ps
http://www.aoc.nrao.edu/~egreisen/scs_050829.pdf
In the text below, lines starting with > are from the various
reviewers including Mark Calabretta while my responses do not start
with "> ".
Thanks for all your work,
Eric Greisen
****************************************************************************
Comments forwarded by Pence from IAU reviewers (omitting those for
which no response other than "thank you" seems needed).
> I realize I am reacting rather late, but I find that the multi-dimensional
> -TAB does not really belong with the rest of the paper. It is undoubtedly
> useful, but it has very little to do with spectral coordinates. If it were
> a simple addition I would not worry about that, but by itself it is
> responsible for about half the complexity of the paper which is not
> simple. I am still voting yes because I don't want to hold the process
> (and out of guilt for not checking earlier ;-)), but I'd really like the
> authors to consider splitting this part to a separate paper or moving it
> to paper IV, leaving only the simple case of a single axis in paper III.
-LOG, -TAB, CNAMEia, and perhaps also the general mathematical method
described in Sect. 3, are all quite comfortably Paper I issues, so
if/when the WCS standard is ever collated into a NOST-like document it
would make sense to depart from the historical order. These matters
were driven by spectroscopic issues, but the solutions we have
proposed are more general and so should have appeared in Paper I had
we been willing to delay that paper even longer.
> Very minor detail, but I'd suggest replacing "Snell law's of refraction"
> by "The Snell-Descartes law of refraction". I don't quite remember which of
> them actually had the legitimate prior claim, but some fraction of the
> readership learned the refraction law under Descartes' name and will scratch
> their heads on Snell's law
A footnote has been added.
==========================================================================
> I read Paper III and, in consideration of the large effort spent (and
> competence shown) by the authors, I decided to give a favorable vote,
> although I feel the paper is rather specific and difficult (and/or I am
> rather incompetent in the matter ! ... which would have called for an
> abstention).
>
> I have two general orders of comments (non-technical).
>
> One is that, as said, the paper is quite specific (mainly to technical
> details of "optical" spectroscopy) and difficult (also because it aims to
> handle general mechanisms), and somehow lacks of two things : one are
> COMPLETE examples (I mean e.g. including FULL header examples - but see
> what the referee says on this formal aspect). The other thing is that is
> not in a prescriptive form, it does not provide a step-by-step recipe to
> writing headers ... but possibly this is not a matter for a paper, but
> more for a "WCS cookbook" (could this be an idea for the FWG or some
> dedicated task force when reviewing manuals and guidebooks ?).
I am surprised by the comment that the paper leans toward optical
spectroscopy matters; the usual remarks have been that the paper
contained too much radio spectroscopy which I understand much better.
The authors do agree that a cookbook or Users' Guide is needed not
only for WCS issues but also for all other issues in the FITS format.
> The second order of comments stems in part from the fact I've so far
> considered the WCS (or the "2-d WCS" in Paper I and II) as a sort of
> "black box" (and paid no attention to what is inside) to describe
> "projections" for data where the distinction between raw and reduced data
> is blurred, while, on the other hand, pixel-by-pixel re-projection is
> complex or not worthwhile.
>
> In this respect I found very useful (and up to my expectations as "user")
> to be able to have tools which e.g. get sky coordinates clicking on a
> pixel image "simply" writing the proper header kwds and not modifying the
> image itself.
>
> For spectra however my expectations where different. I'm inclined to make
> a stronger distinction between "raw" data (the IMAGE on the focal plane of
> a spectrometer, for the few cases where I had direct experience, limited
> essentially to old IUE low dispersion spectra, and to VIMOS multi-object
> spectra) and "reduced" data (essentially a - calibrated - spectrum seen as
> a TABLE with flux vs energy - or frequency or wavelength). OK ... the
> table could also be a 1-d image, but anyhow I'd expected it to be a
> reprojection and rebinning and reprocessing of the raw data.
>
> So I'd expected something quite simpler like essentially a list of
> convention on physical units and scales (lin, log, sparse or tabular),
> which could be used e.g. to make composite spectra from multi-wavelength
> observations (from the REDUCED data of such observations).
It is the goal of our work to represent both raw and reduced data of
all sorts whenever possible. Clearly, some raw data are so non-linear
that they will be hard to represent until Paper IV methods become
available. Other raw data require interpretation before world
coordinates can even be described for the data pixels. I am thinking
of images taken through a disperser which cause the same RA-Dec to
occur at multiple places in the raw data depending on the wavelength.
> I see in such respect no reference at all e.g. to high-energy count or
> photon spectra ... even from
> the trivial fact that customary "keV" are missing as "preferred" energy
> units in Tab. 1. I noticed you [PENCE] did not make any specific comment
> on behalf of the high energy community. Are you (also as HEASARC) thinking
> somehow of using things like -TAB to describe X-ray PHA or PI spectra, or
> instead want to leave them alone and consider them something "totally
> different" ?
The issue of units is dealt with in Paper I and includes both kilo and
ev.
> Another (partially related) issue I did not see (and I could have
> expected), is handling of "error bars" or "large bins" .... think of the
> lower and upper channel boundaries in a photon spectrum and the way to
> properly convert them from dN/dE to FE, Fnu or Flambda, also using a
> spectral model. But the same may apply to any "broad band" spectra (either
> rebinned or simply derived from photometric points).
>
> Now I find in paper III something DIFFERENT, which is clearly more
> consistent with the "philosophy" of the "image" papers (leave the raw
> data intact and provide a way to read world coordinates point-by-point),
> and also addresses problematics specific of other frequency domains and
> communities (see emphasis on "velocities", "air wavelengths" or "grisms").
The issue of uncertainties in coordinates is partially addressed by
the new keywords CRDERia and CSYERia of Paper I. The issues of the
binning of each pixel or, more generally, the spectral, spatial, and
temporal resolution of the data have not been addressed in any of
these papers. Most images are blurred by a point-spread function so
that adjacent pixels are not independent and this also occurs in the
spectroscopic domain. Although this is very relevant to the physical
and statistical interpretation of the data, it does not affect the
coordinates. (Actually this may not be precisely true. For example,
if the spectral response across a spectral channel (bin) is not
symmetric, then the actual average frequency would not be the center
of that bin.) This is an interesting issue and perhaps someone would
like to take it up in yet another FITS paper.
When a cube is constructed of images at different frequencies to
define an SED, then any assumption that coordinates may be
interpolated between pixels is incorrect. Section 6.2.3 illustrates
how the -TAB "projection" may be used to deal with this issue.
> None of the above considerations is however such to deny a favorable vote
> on the content of Paper III, which is likely to be very valuable for the
> communities for which it was intended.
======================================================================
> The solution in chapter 6 for non-linear non-separable coordinates,
> which occur very often in spectroscopy (particularly IR spectroscopes
> which are compact), is very complicated. A
> "distortion correction" possibly as described in "Paper IV" is highly
> desirable for a workable solution, and should be given a
> high priority.
We agree.
> The suggested formulae for refractive indices in Chapter 4 are
> inappropriate for IR wavelengths > 2microns, which are in
> common use.
********* Information added by Walter Jaffee ********************
>> Changes in refractive index are usually caused by absorption lines to
>> the blue of the wavelength of interest. For the optical these are all
>> in the UV, so one formula can fit the whole optical band. The various
>> atmospheric absorption lines that separate the IR bands (I,H,J,K,...)
>> each causes strongly curved refractivity changes in the immediately
>> adjacent bands.
>>
>> So it is probably best to specify a formula per band, rather than
>> expect one formula to cover all the bands with acceptable accuracy.
>>
>> Another complication is that several of these lines are caused by
>> water vapor, so that one has to specify not only the dry air density
>> (or pressure/temperature) but also the humidity in order to adjust a
>> formula to the specific observing conditions.
>>
>> Richard Mathar, who now works for me in Leiden, is one of the experts
>> in calculating these curves. c.f. Mathar, R., 2004 Applied Optics, 43, 928.
********* End Information added by Walter Jaffee ********************
I have added a footnote saying just about what you have. The
refractivity at UV and IR sounds way too complex for the present
paper.
> In another context, it would be desirable to be able to specify the
> "true" wavelength (freq,velocity..) coverage of each pixel, i.e.
> the spread of wavelengths contributing to the flux.
> It is commonly assumed by naive users that this equals the coordinate
> difference with the preceding and following pixels, but this
> is seldom the case. This is of course equivalent to specifying the
> "resolution" of an image, which is very different from the pixel
> spacing.
Agreed, but beyond the scope of WCS Papers I - IV; see above.
======================================================================
> I think the simpler forms of the -TAB table lookup capability will
> be popular. I would have preferred to see a simpler mechanism than
> currently specified - I suspect the advanced indexing capability will be
> infrequently used. The important case of a spectrum stored as a row of
> a table, with one cell used as the spectral coordinate vector, appears to
> be adequately supported. Another common and important case is a spectral
> data cube where each 2D image is at a different spectral coordinate.
> For cubes with a limited number of spectral planes, ideally one would
> like to describe the spectral axis entirely within the primary image HDU
> (i.e., without requiring a separate table extension), however it did not
> appear to me that that was possible.
We anticipate using the indexing a great deal in radio interferometry.
Table 10 and Figure 7 represent what our data will look like fairly
soon, except that there will be many 1000's of spectral channels.
If the spectral axis is simple, then the standard data header may
easily be used with a suitable CTYPE for spectral cubes. If the
spectral axis is complicated, then a FREQ-TAB header axis may be
required.
======================================================================
> After pondering the FITS paper on spectral coordinate systems, I will
> vote in the affirmative--to accept it as part of the definition of
> the FITS standard. I think the problem domain is defined clearly in
> the paper, it is dealt with thoroughly, and the solutions in terms
> of FITS conventions and techniques are generally elegant and
> occasionally quite clever.
> I have a few concerns though, which I thought I would pass on. The
> first is that this paper is a pretty tough read. While there are a
> lot of technical issues to sort through, I fear that a lot of the
> academic community--and even many of those who would attempt to
> implement the standard in their data systems, will find the paper
> nearly impenetrable. I feel that community standards ought to be
> both enabling and approachable, but to achieve the latter goal will
> almost certainly require a much longer "user" document if we expect
> mere mortals to implement the standard correctly. The subject is deep,
> but the paper does not do enough in my view to make the problems
> simpler to understand for the purpose of explaining the correct
> application of the standard.
>
> The second concern relates to section 6.2.3 on the use of -TAB portion
> of the convention, as it applies to multiple exposures. I find the
> argument for creating a higher-dimensional structure to relate
> sequences of 2-D images to be unpersuasive. The analysis of time-
> sequence images, or of images in different bandpasses of the same
> location, is in my view unlikely to benefit much from the proposed FITS
> conventions in this area (e.g., generation of light curves,
> multi-wavelength modelling of source SEDs, etc.). The major benefit
> identified in the paper is for the case of an image display application,
> where a cursor read-out of coordinates would be perhaps more informative.
> In my view, the suggested convention would add complexity to the standard
> without much benefit. I have a feeling that the relationships between
> images in a sequence would be more effectively dealt with in a VO
> standard, rather than within FITS.
I agree that Section 6.2.3 presents a fairly complicated scheme to
represent the explicit spectral and temporal limits of each plane of
the image cube. It was included to illustrate what can be done with
the -TAB nomenclature to answer in part the concerns several of us
have about also describing the resolution (spectral and spatial) of
the data. This is particularly true when people do construct spectral
cubes to represent SEDs which they will do in a FITS context as well
as the VO context.
> My third concern is really more of a question. It relates to section 7,
> and the definition of the SSYSOBSa keyword & associated values. While
> I believe I understand the argument for making plain the velocity
> reference frame to which the coordinates apply, I'm not sure I follow
> the choice of values for this keyword. By selecting a particular system,
> does that mean that a particular magnitude for a velocity correction has
> been applied to the data? Wouldn't a change to the convention (such as
> the magnitude or direction of CMBDIPOL, as cited in the paper) degrade
> the accuracy of the coordinates represented in those FITS files created prior
> to said change in convention? Clearly the definition of these conventions
> lies outside the control of the FITS standard. If I understand correctly,
> it would be unfortunate for a change to a velocity convention to have
> the side effect of degrading the validity of all FITS files in existence
> at that time, where this portion of the standard is used. A possible
> improvement would be to define yet another keyword that would state
> explicitly the numerical values for the reference velocity vector that is
> being referenced. That way if the velocity convention is changed, it will be
> apparent how to interpret the data files.
> [I readily confess that I may not have understood the issue, and that my
> comment may not make any sense. If so I apologize.]
I doubt that the older reference frames will change - they already
represent rather gross averages and round offs of the observational
data. Perhaps we can take the WMAP, year 1, result in a similar
fashion as a standard and ignore later, presumably subtle, revisions
of their result. The text already advised against use of CMBDIPOLE at
present since it is likely to change. I have added columns to the
table showing the standard directions quoted for the systems and a
well-commented phrase in the text concerning the WMAP system.
****************************************************************************
Mark Calabretta comments and suggestions
Ones not acted upon:
======================
> p2: "Furthermore, the apparent ... rather than the center of mass of
> the object." - I don't think this is relevant, the question of what
> exactly it is that you're measuring the velocity of is not at issue.
In this context I am thinking about an unresolved star whose light
is dominated by something that is moving wrt the star you think
you are observing. A consideration not to be forgotten (although
it usually is).
> p12: I've said it many times before but once more won't hurt: the second
> sentence of Sect. 6.1 doesn't make sense.
I do not understand why you do not understand that sentence. We
were asked by knowledgeable people this very question. I know
that to you it is obvious that a fundamental coordinate must be
what is represented in the table. But others wanted to put
RA---SIN in the table. This sentence is there to say no to that
probably silly idea. I suppose in an image that was close to
-SIN but distorted one might want a simpler table to reflect that
distortion in arc sec rather than translate all the way to RA and
DEC which is why someone asked the question.
> p14: It is not immediately obvious how the example given in the second
> para of Sect. 6.2 would be handled. Suggest
>
> "by a 2-dimensional table lookup." ->
> "by a two-dimensional table lookup using a degenerate axis in a way
> similar to the example described in Sect. 6.2.3."
I had nothing so elaborate in mind. The telescope does traverse
a smooth path. I was simply proposing a 2-d grid of RAs and Decs
without worrying that in pathological cases one track attempting
to be constant Dec might cross the adjacent one. I meant this to
be only a simple example to tell a grid program where to center
the convolving function for each spectrum.
========================================================================
Ones acted upon
A re-wording in Appendix A was supplied along with a re-working of the
equations, making a less formidable and shorter equation set.
> p2: The graphs of Fig. 1 have some very interesting features that no one
> has commented on as far as I am aware. I suggest replacing the last
> sentence of the caption with:
>
> Note that each family of curves intersects the axis of zero velocity
> measure at the same values of $\vv\sub{t}$ because the redshift is
> zero for these combinations of $\vv\sub{t}$ and $\vv\sub{r}$. Thus,
> regardless of the value of $\vv\sub{t}$, for any given redshift the
> velocity measures agree on whether the object {\em appears} to be
> receding or approaching. However, the object may appear to be
> receding at high speed even when it is actually approaching at high
> speed, though the converse is never true. At transverse velocities
> of $\vv\sub{t} / c > \sqrt{2} / 2 (\approx 0.71)$ all of the velocity
> measures are positive (receding) regardless of whether the object is
> actually receding or approaching. Furthermore, for $\vv\sub{r} / c$
> below $-0.5$, the faster the object approaches, the smaller the
> transverse velocity required to make it appear to be receding!
>
> p2: "It may also be affected by ... and by velocities local to the
> "event of observation," such as motions of portions of the
> telescope with respect to the surface of the earth"
>
> The question of the Doppler frame is not at issue, topocentric
> velocities can be converted to another frame. In this context it
> would be far more relevant to mention the cosmic redshift. I
> suggest changing to
>
> "It may also be affected by ... and in particular by the cosmic
> redshift."
>
> p5: Equation (45) is given as dX/dw = (dP/dS) / (dP/dX), rather than
> dX/dw = (dX/dP) * (dP/dS) for an important reason that is not
> mentioned. Remove the full-stop from Eq. (45) and add
>
> "which gives $\D{\vX}/\D{\vw}$ as a function of $\vX\sub{r}$ --
> $\D{\vP}/\D{\vS}$ being constant and $\D{\vP}/\D{\vX}$ a function
> of $\vX$."
>
> p5: "the latter from Eq. (45)." ->
> "the latter obtained from Eq. (45) as a function of $\vX\sub{r}$."
>
> p6: In the first of the unnumbered equations, I originally added the
> last equation in the sequence to assist in deriving the second last
> unnumbered equation at the bottom of the page. However, on working
> through it now I find that it just confuses things. Replace it with
>
> = - \frac{\nu\sub{r}^2}{\nu_0} \cdot
>
> p12: "However, it should not be considered valid" ->
> "However, it is not valid". (There's no two ways about it!)
>
> p13: "The prescription above requires that, if $\Psi_k = \Psi_{k+1}$,
> then $\psi_m = \Psi_k$ and the result is undefined."
>
> On re-reading I found this unclear, change to:
>
> "However, if $\Psi_k = \Psi_{k+1} (= $\psi_m)$ then the result is
> undefined."
>
> p13: "by the keyword TDIMn, where" ->
> "by keyword TDIMn set to $'(M, K_1, K_2, \ldots, K_M)'$, where"
>
> Add following that:
>
> "Note in particular that if $M = 1$ the coordinate array is
> formally {\em two-dimensional} though the first axis is
> degenerate."
>
> (I've now implemented -TAB at the highest level in WCSLIB and this
> turns out to be significant.)
>
> p14: It is not immediately obvious how the example given in the second
> para of Sect. 6.2 would be handled. Suggest
>
> "by a 2-dimensional table lookup." ->
> "by a two-dimensional table lookup using a degenerate axis in a way
> similar to the example described in Sect. 6.2.3."
>
> p18: Surely the default value for SSYSOBSa should be 'TOPOCENT'!
>
> p19: DATE-AVG is more suitable for human consumption and MJD-AVG better
> for computation so I don't think it is appropriate to advise
> against using them together. Thus omit "; thus, it would be better
> not to use the two together".
>
> p22: "12 hours occurs for a field at the north or south point" ->
> "12 hours occurs for an equatorial telescope observing a field at
> the north or south point".
>
> p23: "Two of these are used" ->
> "SPECSYSa, SSYSOBSa, SSYSSRCa, and ZSOURCEa are used".
>
> ", seven enable" ->
> "; MJD-AVG, DATE-AVG, OBSGEO-X/Y/Z, VELOSYSa, and ZSOURCEa record
> parameters for the" (note comma changed to semi-colon).
>
> ", and two define" ->
> "; and RESTFRQa, RESTFREQ, and RESTWAVa define" (ditto).
>
> p23: The first three equations in Appendix A are important enough to be
> numbered.
>
> p23: It is important to state that v_s is non-negative (the sign is
> supplied by theta):
>
> "and $\vv_s$ is the total space velocity" ->
> "and $\vv\sub{s} \ge 0$ is the total space velocity".
>
> p23: I disagree that Eqs. (A.10), (A.12), and (A.13) are of any
> interest. Omit them and then combine Eqs. (A.9) and (A.11),
> unnumbered, as
>
> \sin\theta = \nu ...
>
> = \lambda ...
>
> This will provide logical numbering for equations in Appendix A.
>
> p24: "to express the angle $\theta$" ->
> "with default value $+90\degr$ to express the angle $\theta$".
>
Plus a substantial number of typographical and grammatical
suggestions.
More information about the fitswcs
mailing list