[fitswcs] [forwarded] Version 0.982 of the FITS WCS Time Paper
Patrick P Murphy
pmurphy at nrao.edu
Mon Jun 3 18:32:35 EDT 2013
Due to the default 40KB max on Mailman lists, this didn't get through
the first time. The threshold is now 125K.
- Pat
------- start of forwarded message -------
From: "Rots, Arnold" <arots at cfa.harvard.edu>
To: fitsbits at nrao.edu, fitswcs at nrao.edu
Subject: Version 0.982 of the FITS WCS Time Paper
Date: Mon, 3 Jun 2013 18:00:53 -0400
In response to comments on version 0.981, I have now posted version 0.982:
http://hea-www.harvard.edu/~arots/TimeWCS/
There were three sets of comments, by Mark Calabretta, Bill Pence, and Lucio
Chiappetti.
I reproduce their comments below, with my responses interspersed, marked by
"--->"
I trust we are getting closer ;-)
Cheers,
- Arnold
My response to Mark's comments.
Responses indicated by --->
- Arnold
----- Forwarded message from Mark Calabretta -----
From: Mark Calabretta <mark at calabretta.id.au>
To: fitsbits at nrao.edu, fitswcs at nrao.edu
Date: Fri, 08 Feb 2013 13:46:49 +1100
Subject: [fitsbits] [fitswcs] Version 0.981 of the WCS Time Standard Draft
I have a few comments on the WCS time paper now that everyone has
had ample time to review it.
These have all been discussed previously in private but I thought
they should have wider circulation before any decisions are made.
Regards,
Mark Calabretta
>>>
Sect. 3.4:
The requirements (a) for doublet time values to be split into
integer and fractional parts, and (b) for them to be of the same
sign, seem unnecessary and unwarranted to me. I expect they will be
the first parts of this proposal to be violated in practice.
---> I don't see why this would be the first part to be violated.
The intent is to make the rule as unambiguous and as simple to
implement as possible.
This way the user can handle the individual part as data types
that are standard and fully predictable, moving the
high-precision considerations to the stage where the two get
combined - at the discretion of the implementor.
The same-sign requirement is there specifically to make sure that
there is no misunderstanding in the interpretation of negative values.
Sect. 4.1.1:
How would Julian or Besselian epoch be encoded in a time axis in an
image? A numerical scale in years can easily be constructed, but
how would it be designated?
---> This is a valid question that I sort-of answered previously, but now
I have formulated a solution to this in a new sub section: 6.6.
Sect. 4.1.2:
The handling of MJDREF and JDREF could be improved significantly by
relaxing the current requirement that the high-precision forms be
split into integer and fractional parts.
Instead of having [M]JDREF on the one hand, and [M]JDREFI, [M]JDREFF
on the other (BTW, which takes precedence?), it would be more
natural and easier to implement if there were just [M]JDREF and
[M]JDREFF. Thus [M]JDREF would always be written, and if extra
precision were needed then [M]JDREFF would contain the minor part.
This has several advantages:
1) Eliminates two keywords, [M]JDREFI.
2) Eliminates the question of precedence.
3) Allows users to split [M]JD into high-precision forms as they
see fit, not being constrained to integer+fraction.
4) Easier to parse because there is no dichotomy between the
normal and high-precision forms.
If this conflicts with current usage then the keywords can easily
be renamed.
---> This may have been preferable, but (as stated) we tried to
conform with existing usage whenever possible and I don't think
this proposed change offers sufficient benefit to depart from that.
Sect. 4.1.3:
The values defined for TREFPOS in Table 3 vary from those defined
for SPECSYS and SSYSOBS in Paper III and use US spelling.
By the simple expedient of ignoring all but the first 3 characters,
I propose to allow any of the forms:
TOPO, TOPOCENT, TOPOCENTER, TOPOCENTRE, TOPOCENTRIC
GEO, GEOCENT, GEOCENTER, GEOCENTRE, GEOCENTRIC
BARY, BARYCENT, BARYCENTER, BARYCENTRE, BARYCENTRIC
HELIO, HELIOCENT, HELIOCENTER, HELIOCENTRE, HELIOCENTRIC
EMBARY, EMBARYCENT, EMBARYCENTER, EMBARYCENTRE, EMBARYCENTRIC
GALACTO, GALACTOCENT, GALACTOCENTER, GALACTOCENTRE, GALACTOCENTRIC,
GALACTIC, GALAXY
---> OK
Sect. 4.1.4:
The recursively defined keyword, TREFDIR, is outside the scope of
current FITS header parsers and is unnecessary when a pair of
keywords, TREFRA and TREFDEC, would serve the purpose adequately in
image headers.
In binary tables, I can see that there may be some justification for
using an indirection and there is even a precedent for that in
coordinate system cross-references, defined in Sect. 3.5 of Paper I.
However, I would also point out that this is one of the few parts of
Papers I, II, and III never to have been implemented in software, at
least not to my knowledge.
The point of TRDIRna is to avoid duplicating columns and this could
also be achieved with a mechanism to give multiple names (TTYPEn) to
a table column. In the example header of Table 10, the 'EventRA'
column would be given an alternate name, 'TRRA20', and likewise
'TRDEC20' for the declination column. TRDIR20 could then be
dispensed with.
---> This is something that has been discussed before.
The primary application is certainly binary tables and two things
need to be kept in mind: TREFDIR coordinates may not be expressed
in equatorial coordinates (which is a problem in Mark's
alternative) and Bill (see response to his comments) is wrong in
assuming that a single direction can replace a column.
Sect. 4.2:
ta and Ba do not conform to the notion of units introduced in
Paper I as being constant multiples of some basic unit - metres,
seconds, degrees, etc. Software handles units by converting to this
basic set and this is not possible for ta nor Ba as defined here
because the conversion factor varies with time.
A constant conversion factor is required for software interpretation
of legacy FITS files, e.g.
1 Ba = 365.242198781 d, and
1 ta = 365.242190402 d.
This proposal should define constant factors that best suit generic
purposes. Software that applies them should issue a warning.
---> This also has been answered before. We caution against the use of
these units, but the reality is that they have been used. P{art
of the problem is that people have used different values for
these units in the literature. What we provide is the most
accurate value, with the explicit warning that, when these units
are used, there is no guarantee that the author used these
values. We also provide an alternative value as the most accurate
constant value that has been in use.
So, I want to stick with the text as is.
Sect. 4.3.1:
"It is sometimes convenient to be able to apply a uniform clock
correction in bulk by just putting that number in a single keyword."
A global time reference point is already defined, and in triplicate
for good measure: MJDREF, JDREF, and DATEREF. Two of these also
have split forms: MJDREFI, MJDREFF, JDREFI, and JDREFF.
Given that a clock correction can be applied via one of these, or
via CRVALia or CRPIXia, what possible justification can there be for
introducing yet another mechanism, TIMEOFFS?
This multiplicity of ways of doing what are essentially trivial
operations just makes it easier to get things wrong.
At the least, TIMEOFFS should not apply to image axes.
---> I can go along with not allowing its application to image axes.
Sects. 4.3.4 and 4.3.5:
As for TIMEOFFS, these two sections should be revised so that
neither TIMEDEL nor TIMEPIXR apply to image axes.
In terms of sampling theory, the elements in a FITS data array are
obtained by convolving the quantity being measured with the
instrumental broadening function and then sampling with a set of
delta functions on a regular grid. When presenting the data array
visually as an image, the WCS standard essentially says that pixels
should be displayed so that their centre corresponds to the
coordinates recorded for the delta functions. In other words, the
set of delta functions is convolved with a top-hat function - a
"pixel" - which is centred at the origin rather than offset from it.
Saying that a world coordinate occurs anywhere other than the pixel
centre is equivalent to saying either that the instrumental function
or the top-hat function are offset from the origin, or that the
coordinates recorded for the delta functions have a constant offset
(zero error). In either case, the proper way to address this error
is to correct the pixel and/or world coordinates by adjusting
CRPIXia and/or CRVALia. Thus, in this context, there is no reason
nor justification for introducing a new keyword such as TIMEPIXR
that enshrines the offset as a separate entity.
The statement that "clock readings are effectively truncations, not
rounded values, and therefore correspond to the lower bound of the
pixel" makes no sense to me. The same could be said of the encoders
that read telescope position. However, one assumes that
measurements are taken with sufficient precision that truncation has
no effect. Even if it did the values of CRPIXia and/or CRVALia can
be tweaked to compensate.
On the other hand, in a situation where a separate time is recorded
for each of a set of time bins, it may well be that the time refers
to the start, middle, or end of the sampling period. In that case
introducing a keyword like TIMEPIXR might be justified to save
correcting each of the many times recorded. However, it is not
explained how TIMEPIXR is to be associated with such time values.
Therefore, I feel strongly that TIMEPIXR should only apply in this
latter case and have no interaction with image axes - CRPIXia,
CDELTia, and the binary table forms.
---> OK, not for image axes
With regard to TIMEDEL, "resolution" is usually taken as the width
of the instrumental sampling function, not the spacing between the
sampling delta functions. It is unclear how TIMEDEL is meant to be
used, in particular, how it interacts with image axes.
---> As time resolution or, if you like, the uncertainty in the time
stamps.
------------------------------
-----------------------------------------
The following are suggestions for improving the paper itself, rather
than the proposal.
General:
The particular words, "may", "shall", "must", etc. are used with
particular meanings. Those meanings should be defined.
---> I suppose we can do that, though I don't remember seeing that in
any of the previous papers.
Abstract:
So far, four WCS papers have been published as FITS standards.
"World coordinate functions are defined... as specified by a lookup
table."
What functions? What lookup tables?
---> This text may originally have come from Peter.
Coordinates and transformations - in images and in (binary) tables.
Sect. 1:
"The prescriptive part of this standard is contained in Sections 2,
3, 4, and Appendix A."
Section 2 only contains the terms of reference which are not
prescriptive.
---> I still think it should be part of standard, for the same reasons
as those Lucio gives.
Conversely, Sect. 6.2.1 is prescriptive, but probably should be
moved somewhere into Sect. 4.
---> One could say the same about section 6.3.1 and 6.4. These are
explicit expressions (in specific usage contexts) of what is said
elsewhere. I don't mind designating them as prescriptive, but it
is very hard to move any of these to Section 4.
Sect. 3.1:
"In this paper we will use datetime as a pseudo data type to indicate
its use. Following the current FITS standard (Pence et al., 2010)
its values must be written as a character string in A format, but if
and when an ISO-8601 data type is adopted, it should be used in
table columns, rather than the string type."
If an ISO-8601 data type is ever going to be defined (by and for
FITS) then this paper is the place to do it.
---> No, that would really delay this.
It could be as simple
as an array of six floating point values, or, for compactness, an
array of 12 bytes interpreted as five integers and a float as
follows:
bytes 1-4: year (int, big-endian)
5: month (unsigned char, 1-12)
6: day of month (unsigned char, 1-31)
7: hour (unsigned char, 0-23)
8: min (unsigned char, 0-60)
9-12: second (float, IEEE big-endian)
---> I think there are better ways (see also Lucio's comments), but I
don't think it is relevant at this point.
Sect. 4.1.1:
The equation for T(TDB) in terms of T(TCB) is missing a factor of
86400, it should be
T(TDB) = T(TCB) - L_B (JD(TCB) - JD0) * 86400 + TDB0
Also, units should be given for TDB0, i.e.
TDB0 = -6.55e-5 s
--> Agreed; the same is true for the equation for T(TCG).
Sect. 4.1.2:
The obvious question that should be answered explicitly in this
section is that of representing one of the time scales in Table 2 as
an MJD or JD.
---> I have no idea what this means.
Sect. 4.1.3:
OBSGEO{X,Y,Z} are not being defined here so shouldn't receive
special formatting.
---> Answered before. Though these were defined in a previous paper,
for the purposes of this paper they are at the same level as the
newly defined keywords. The question is whether this formatting
should strictly convey what is original or what are specific
keywords defining essential parts of the time WCS. I choose for
the latter, since I suspect the subtle difference may be lost on
the reader.
Sect. 4.5:
A better distinction to draw between FREQ (as per Paper III) and
FREQUENCY would be that FREQ applies to electro-magnetic waveforms
of cosmic origin with fixed transformations to related variables
such as wavelength, velocity, and redshift which do not apply to
periodic phenomena in general (not even terrestrial EM waves).
---> Or should we add this as an alternative distincting definition?
Sect. 6.2 and Table 6:
The header in Table 6 is missing the CDELT3A keyword which therefore
incorrectly defaults to 1.0.
---> Correct
There is no explanation or discussion of this header in the text
when there are certainly some useful points that could be made about
---> Is it really needed?
it. E.g. how the time axis might be represented as an MJD, or the
fixed relationship between CRVAL3 and CRVAL3A. Conversely, the
celestial coordinates aspect of the header is overly complicated for
an example that is meant to illustrate time coordinates.
---> Maybe, but it is what it is.
I will move the CDELT3 keywords up in the header to make this
clearer.
Table 7:
It is a requirement of the WCS standard that CUNITia be 'deg' for
all celestial axes and this has never been relaxed. Therefore this
header is non-standard!
---> Also answered previously. This is just the way it is done in the
solar community and deviating from that is not going to be
helpful. See also Bill Thompson's earlier response to this
comment.
Table 8:
Considering that Table 8 differs from Table 7 only in six keyvalues,
it would be preferable just to list those six in the text.
---> I raised this question several times, but received no answer,
except from Mark, so I figured people were content to leave it
in.
Sect. 6.2.3 and Table 9:
The rather large header in Table 9 only demonstrates that an image
can have two coordinate descriptions which by now should be well
understood. Moreover, it is unrealistic to expect that anything
other than special-purpose software could do anything meaningful
with it. In fact, it would be misleading to most WCS interpreters
which would ascribe both coordinate systems to every frame.
Therefore I think it unsuitable for inclusion in this paper.
The proper way to handle this situation is via the binary table as
stated later, contradicting the statement made beforehand that "it
is not possible for a FITS file to store a complete description of
the WCS for every frame in a movie" which should be removed. Using
a binary table seems a perfectly good solution to me, which begs the
question of why the example header was not based on that approach.
---> Well, it is what it is. Argue with Steve Allen who contributed
this example. I could add: "within the context of a single HDU".
Table 11:
Example 6 shows a binary table purporting to be a pixel list in
which all world coordinates are recorded directly. But in that case
the WCS pixel list formalism is not needed! It should ring alarm
bells that the Time, Phase_1, and Phase_2 columns are all simple
linear functions. The whole point of pixel lists is that pixel
(e.g. detector) coordinates are recorded and the WCS keywords say
how to convert them to world coordinates. In this case the Time
column can be considered to be the "pixel" coordinate, the trivial
WCS for column 1 then simply gives units of time to its nominally
dimensionless values. The Phase_1 and Phase_2 columns can be
dispensed with, replacing their WCS keywords with alternate
descriptions of the first column, say 'A' and 'B', as follows:
TCNAM1A = 'Phase of feature 1'
TCTYP1A = 'PHASE'
TCUNI1A = 'turn'
TCRPX1A = 0.0
TCRVL1A = 0.0
TCDLT1A = 0.000605327 / = 1/1652.0
TCNAM1B = 'Phase of feature 2'
TCTYP1B = 'PHASE'
TCUNI1B = 'turn'
TCRPX1B = 0.0
TCRVL1B = 0.75
TCDLT1B = 0.000302663 / = 1/3304.0
Note here that phase is allowed to increase to 1.0 and beyond. It
is arguable whether the TCZPHna or TCPERna keywords are necessary
here as both the zero point and period may be obtained trivially
from the linear transformation parameters.
---> My response is, once again: it is what it is.
I think the example is still illuminating, although things could
be done differently (which is true for any example). Or should
Mark's description be provided as a separate example, as an
alternative to example 6? (like Table 8 is an alternative for
Table 7)
-----------------------------------------------------------------------
Typos, grammatical errors, etc.
Abstract:
"In a series of four previous papers... This fourth paper..."
^^^^ ^^^^^^
"Time on all scales and precision known... shall be described"
^^^^^^^^^ ^^^^^
->
"Time on all scales and precisions known... will be described"
(Use of the word "shall" is grammatically incorrect here.)
Sect. 1:
"the precision which can be" -> "the precision that can be"
^^^^^
"implemnting" -> "implementing"
----- End of forwarded message from Mark Calabretta -----
-------------------------------------------------------------------------------------------------------
============================================================
My responses to Bill, marked by --->
- Arnold
----- Forwarded message from William Pence -----
Date: Fri, 8 Feb 2013 13:29:17 -0500
From: William Pence <William.Pence at nasa.gov>
Subject: Re: [fitsbits] [fitswcs] Version 0.981 of the WCS Time Standard
Draft
I agree with most of Mark's comments:
On 02/07/2013 09:46 PM, Mark Calabretta wrote:
>
> I have a few comments on the WCS time paper now that everyone has
> had ample time to review it.
...
>
> Sect. 3.4:
> The requirements (a) for doublet time values to be split into
> integer and fractional parts, and (b) for them to be of the same
> sign, seem unnecessary and unwarranted to me. I expect they will be
> the first parts of this proposal to be violated in practice.
I suggest changing the text so that it merely *recommends* that both
have the same sign to avoid ambiguities. It is just common sense to
represent the value -3.123456789
as -3.0 plus -0.123456789 rather than -4.0 plus +0.87654321, but I don't
think this is so critical that we must forbid the use of mixed signs.
Who knows, someone may actually think of a good reason to have mixed
signs as a way of encoding additional information in the FITS file.
There are other well known cases in the past where slight ambiguities in
the FITS Standard have been used to good purpose, so I don't think we
need to so tightly constrain this convention to have the same sign
unless there are good technical reasons to do so.
---> See my response to Mark. Just recommending this usage is useless,
because it does not simplify implementation, nor does it prevent
ambiguity.
...
> Sect. 4.1.2:
> The handling of MJDREF and JDREF could be improved significantly by
> relaxing the current requirement that the high-precision forms be
> split into integer and fractional parts.
>
> Instead of having [M]JDREF on the one hand, and [M]JDREFI, [M]JDREFF
> on the other (BTW, which takes precedence?), it would be more
> natural and easier to implement if there were just [M]JDREF and
> [M]JDREFF. Thus [M]JDREF would always be written, and if extra
> precision were needed then [M]JDREFF would contain the minor part.
>
> This has several advantages:
> 1) Eliminates two keywords, [M]JDREFI.
> 2) Eliminates the question of precedence.
> 3) Allows users to split [M]JD into high-precision forms as they
> see fit, not being constrained to integer+fraction.
> 4) Easier to parse because there is no dichotomy between the
> normal and high-precision forms.
>
> If this conflicts with current usage then the keywords can easily
> be renamed.
The X-Ray community has used this FITS convention for a couple decades,
so it would be dangerous to reuse [M]JDREFI or [M]JDREFF with a
different meaning. I believe the usual interpretations is that the
[M]JDREFI and [M]JDREFF keywords (both must be present) trump [M]JDREF
if it is also present.
---> I agree
> Sect. 4.1.3:
> The values defined for TREFPOS in Table 3 vary from those defined
> for SPECSYS and SSYSOBS in Paper III and use US spelling.
>
> By the simple expedient of ignoring all but the first 3 characters,
> I propose to allow any of the forms:
>
> TOPO, TOPOCENT, TOPOCENTER, TOPOCENTRE, TOPOCENTRIC
> GEO, GEOCENT, GEOCENTER, GEOCENTRE, GEOCENTRIC
> BARY, BARYCENT, BARYCENTER, BARYCENTRE, BARYCENTRIC
> HELIO, HELIOCENT, HELIOCENTER, HELIOCENTRE, HELIOCENTRIC
> EMBARY, EMBARYCENT, EMBARYCENTER, EMBARYCENTRE, EMBARYCENTRIC
> GALACTO, GALACTOCENT, GALACTOCENTER, GALACTOCENTRE, GALACTOCENTRIC,
> GALACTIC, GALAXY
I like this idea of only treating the first 3 characters as significant
and ignoring any optional trailing characters. The only (unlikely)
downside is if we later want to support a new value that happens to
begin with the same 3 letters that are already taken.
---> I am OK with this.
Where it could run into ambiguities is if we were to allow
additional definitions of planetary centers; Table 3 is very
specific, but alternative definitions may become relevant.
However, if and when that occurs, we can make sure we define a
convention.
> Sect. 4.1.4:
> The recursively defined keyword, TREFDIR, is outside the scope of
> current FITS header parsers and is unnecessary when a pair of
> keywords, TREFRA and TREFDEC, would serve the purpose adequately in
> image headers.
I agree that defining 2 keywords such as TREFLONG and TREFLAT seems like
a simpler solution in the case of images, and also for most tables.
Only tables that have multiple time columns might then require the
TREFDIRn keyword to override TREFLONG and TREFLAT. The slight downside
to this is that the values of TREFLONG and TREFLAT would likely
duplicated the value of an already existing pair of mission-specific
keywords, but this doesn't seem like a big issue. Presumably, one does
not need great precision in the TREFLONG and TREFLAT values for their
use in calculating pathlength delays so it would usually not be a
problem if the values of the mission-specific pair of keywords are
subsequently updated slightly, and causing them to not exactly agree
with the TREFLONG and TREFLAT values.
---> You underestimate the effect: even for 1 arcmin offset the error
may already run in the milliseconds (< 10 ms in a worst case).
> In binary tables, I can see that there may be some justification for
> using an indirection and there is even a precedent for that in
> coordinate system cross-references, defined in Sect. 3.5 of Paper I.
> However, I would also point out that this is one of the few parts of
> Papers I, II, and III never to have been implemented in software, at
> least not to my knowledge.
>
> The point of TRDIRna is to avoid duplicating columns and this could
> also be achieved with a mechanism to give multiple names (TTYPEn) to
> a table column. In the example header of Table 10, the 'EventRA'
> column would be given an alternate name, 'TRRA20', and likewise
> 'TRDEC20' for the declination column. TRDIR20 could then be
> dispensed with.
Providing a mechanism to assign aliases to a column name is an
interesting idea, but it seems out of scope to include such a mechanism
this TIME WCS paper. Preferably, this should be proposed as a separate
new FITS convention that could then be discussed on it's own merits.
---> I think it is needed here, since there is no satisfactory
alternative.
...
> Sect. 4.3.1:
> "It is sometimes convenient to be able to apply a uniform clock
> correction in bulk by just putting that number in a single keyword."
>
> A global time reference point is already defined, and in triplicate
> for good measure: MJDREF, JDREF, and DATEREF. Two of these also
> have split forms: MJDREFI, MJDREFF, JDREFI, and JDREFF.
>
> Given that a clock correction can be applied via one of these, or
> via CRVALia or CRPIXia, what possible justification can there be for
> introducing yet another mechanism, TIMEOFFS?
>
> This multiplicity of ways of doing what are essentially trivial
> operations just makes it easier to get things wrong.
>
> At the least, TIMEOFFS should not apply to image axes.
Restricting TIMEOFFS for use only with tables seems reasonable to me.
---> OK
> Sects. 4.3.4 and 4.3.5:
> As for TIMEOFFS, these two sections should be revised so that
> neither TIMEDEL nor TIMEPIXR apply to image axes.
...
> Therefore, I feel strongly that TIMEPIXR should only apply in this
> latter case and have no interaction with image axes - CRPIXia,
> CDELTia, and the binary table forms.
I agree, unless there is more justification for why these keywords are
needed in the image case.
---> OK
...
>
> -----------------------------------------------------------------------
> The following are suggestions for improving the paper itself, rather
> than the proposal.
Most of these suggestions seem sensible to me. Just one specific comment:
> Sect. 3.1:
> "In this paper we will use datetime as a pseudo data type to indicate
> its use. Following the current FITS standard (Pence et al., 2010)
> its values must be written as a character string in A format, but if
> and when an ISO-8601 data type is adopted, it should be used in
> table columns, rather than the string type."
I think these "if and when" type statements have no place in a reference
standards document. The document can only spell out the current
definition of the standard. It cannot speculate about possible future
changes, or how one "should" interpret the current standard if revisions
are made at some future time. It will be entirely up to the body that
controls the FITS standard to define recommended practice if and when it
makes any changes to the Standard in the future. I think the presence
of these "if and when" statements in the paper serve no useful purpose
and will lead to continued diversionary debate as we proceed through the
formal approval process of this FITS paper, and therefore should be deleted.
---> I think that "if and when" statements are perfectly harmless and
perfectly adequate in case the condition gets met. They serve a
useful purpose in that this standard does not need to be amended
if and when these types get adopted: if they don't nothing
happens; if they do this is what you would want to say. So I
really want to keep them in.
If one wants to propose a new datetime data format for FITS, then I
think this should be done separately so as to not further delay the
completion of this Time WCS paper.
---> Agreed
regards,
Bill
--
______________________________
______________________________________
Dr. William Pence William.Pence at nasa.gov
NASA/GSFC Code 662 HEASARC +1-301-286-4599 (voice)
Greenbelt MD 20771 +1-301-286-1684 (fax)
----- End of forwarded message from William Pence -----
[cleardot]
-------------------------------------------------------------------------------------------------------
============================================================
My responses to Lucio's comments, marked by --->
- Arnold
----- Forwarded message from Lucio Chiappetti -----
Date: Wed, 13 Feb 2013 16:12:16 +0100 (CET)
From: Lucio Chiappetti <lucio at lambrate.inaf.it>
To: FITS WCS mailing list <fitswcs at nrao.edu>
Subject: Re: [fitswcs] [fitsbits] Version 0.981 of the WCS Time Standard
Draft
It is my turn to comment on the latest draft of the WCS Time paper. I
agree with most of the comments made recently, in particular the typos and
formal matters which have been spotted.
I'll intervene only about a few (most already mentioned), starting from
the less important. The first two are my own, the other depend on comments
from Mark and/or Bill.
------------------------------
--------------------------------------
*) abstract and section 1
the issue of the "four" WCS papers of which this is the "fifth" has
already been addressed. The papers "incorporated by reference" are
indeed four (section 8 of the standard), however the HEALPIX paper
has never been formally numbered as "WCS Paper IV" ... but it looks
like the numbering has been de facto abandoned.
---> I have no preference for any kind of numbering. I think I called
this one currently IV.
--------------------------------------------------------------------
*) (status of) section 4.7
the GTI "convention" is a sensible and widely used one. However it
belongs more to the "convention registry" than to the standard itself
(if we keep to the practice followed so far ... if there are good
reasons to deviate, and make it a convention "more equal" than the
other, please talk now ... otherwise I'm afraid it should be moved
to the non-prescriptive part of the paper)
---> I think it rises above the level of a convention, since it has
universal applicability and would be useful as a part of the
standard. So, yes, I made it part of teh standard on purpose -
like XPOSURE is, for instance.
--------------------------------------------------------------------
On Fri, 8 Feb 2013, William Pence wrote: (marked WP)
On Fri, 8 Feb 2013, Mark Calabretta wrote: (marked MC)
*) formalities about which parts of the standard
MC> Sect. 1:
MC> "The prescriptive part of this standard is contained in Sections 2,
MC> 3, 4, and Appendix A."
MC>
MC> Section 2 only contains the terms of reference which are not
MC> prescriptive.
MC>
MC> Conversely, Sect. 6.2.1 is prescriptive, but probably should be
MC> moved somewhere into Sect. 4.
section 2 is however necessary for the understanding of the truly
prescriptive sections, so fits nicely "in the standard".
---> Agreed
I interpreted 6.2.1 as a mere summary of statements made previously,
but if not then should be moved
---> I see it as a summary. However, if this is deemed prescriptive
(and what about 6.3.1 and 6.4?), I feel it is very hard to move
it without loss of clarity.
--------------------------------------------------------------------
*) Sect. 4.1.3:
MC> The values defined for TREFPOS in Table 3 vary from those defined
MC> for SPECSYS and SSYSOBS in Paper III and use US spelling.
MC>
MC> By the simple expedient of ignoring all but the first 3 characters,
[...]
WP> I like this idea of only treating the first 3 characters as significant
WP> and ignoring any optional trailing characters. [...]
I like the three-letter coding too.
---> OK
BTW, should we make them case-insensitive ?
---> I hesitate; comments?
--------------------------------------------------------------------
*) Sect. 3.4:
MC> The requirements (a) for doublet time values to be split into
MC> integer and fractional parts, and (b) for them to be of the same
MC> sign, seem unnecessary and unwarranted to me. I expect they will
MC> be the first parts of this proposal to be violated in practice.
I'm quite surprised by this statement
---> So am I
WP> I suggest changing the text so that it merely *recommends* that both
WP> have the same sign to avoid ambiguities. It is just common sense to
WP> represent the value -3.123456789 as -3.0 plus -0.123456789 rather than
WP> -4.0 plus +0.87654321,
It is like the case of declination ... 5:59:59 = 5.9997(2) deg
and -5:59:59 = -5.9997(2) deg
Since all times are *relative* to MJDREF/JDREF/DATEREF, they can in
principle be either on the positive or on the negative side of the
reference time (as the CRPIX CRVAL may even be outside an image),
although I'd personally chosen a more restrictive way (like having the
reference time equal to 0 UT of the same day of the first time bin in
a light curve, so that all relative times were on the positive side).
---> UT???
But making the integer and fractional parts with different signs
seems just confusing to me.
---> Agreed
The fact that two elements of a doublet are the integer and fractional
parts (*in the chosen unit*) is a different story. Personally I would
have considered times as integer counters with 2**-n s resolution on
the LSB ... but I would have split them so that the most significant
element had 2**0 = 1 s resolution, i.e. effectively integer seconds
and fractions of seconds. Having an arbitrary split elsewhere will be
... just arbitrary !
---> Yes, but consistent with MJDREF[IF]
I am NOT asking to reconsider (from pair of doubles to pair of
integers). I am just noticing there are many possible ways of
handling times (and possibly different ones have actually been
used in the past) ... but the purpose of defining a standard
(badly needed since a long time !) should be a way of cutting
down the possibilities and TAKING A CHOICE. So I am contented
with 0.981 as it stands.
Then there is the issue of "Once FITS forever FITS" ... but
defining the (new) standard will not invalidate existing files
*as FITS files* ... simply they would not be consistent with
FITS time files created after acceptance of the standard.
--------------------------------------------------------------------
*) Sect. 4.1.2:
MC> The handling of MJDREF and JDREF could be improved significantly by
MC> relaxing the current requirement that the high-precision forms be
MC> split into integer and fractional parts.
[...]
MC> If this conflicts with current usage then the keywords can easily
MC> be renamed.
WP> The X-Ray community has used this FITS convention for a couple
WP> decades, so it would be dangerous to reuse [M]JDREFI or [M]JDREFF with
WP> a different meaning.
In principle Mark is correct. The "loosely-typed" nature of FITS
keywords, which are not linked to a strict association to a REAL*4
or REAL*8 should allow to define a date of arbitrary precision e.g.
JDREF = 2456337.126527778
---> But, as argued before by someone, this would require special
handling of specific keywords which is undesirable.
In practice however the reader should probably parse the value
into integer and fractional parts before using it. So both ways
(JDREF vs JDREFI/JDREFF) are ultimately equivalent.
It might be the case to force one among many possibilities, as
said above, but one has to consider whether this violates the
OFAF principle, or simply if this might be an inconvenient for
a large number of existing files ...
--------------------------------------------------------------------
*) Sect. 4.3.1:
MC> "It is sometimes convenient to be able to apply a uniform clock
MC> correction in bulk by just putting that number in a single
MC> keyword."
MC>
MC> A global time reference point is already defined, and in
MC> triplicate for good measure: MJDREF, JDREF, and DATEREF.
[...]
MC> This multiplicity of ways of doing what are essentially trivial
MC> operations just makes it easier to get things wrong.
I tend to agree with Mark ... but I'm sorry this is not of much use
for the paper :-)
---> I consider this benign; there are more dangerous ways to get
things wrong.
--------------------------------------------------------------------
*) Sect. 3.1:
two different sorts of argument under this heading ...
... the first is about a "datetime" type per se
... the second about "if and when" statements
MC> "In this paper we will use datetime as a pseudo data type to
MC> indicate its use. Following the current FITS standard (Pence et
MC> al., 2010) its values must be written as a character string in A
MC> format, but if and when an ISO-8601 data type is adopted, it
MC> should be used in table columns, rather than the string type."
MC> If an ISO-8601 data type is ever going to be defined (by and for
MC> FITS) then this paper is the place to do it.
[then continues proposing examples like array of 6 reals, or 5
integers and 1 real]
I've used myself 7-integer arrays y m d h m s f (with fraction
in 1/100 s) as a form of time *representation* but I would not
have considered them as a way of "internal representation" or
"binary representation", not more than I would consider BCD
instead of IEEE floats :-)
My own internal choice at the time was "Unix 1970" (Posix)
time (which has the known shortcomings described in 4.1.1
of the paper), as reference times. And elapsed 2**-n s from
the reference for individual times.
A running (unsegmented ?) sequence of seconds is a suitable binary
representation for "datetimes" for purposes not requiring
sub-second resolution (timestamps in databases).
So in principle in FITS we would have needed only one new
(binary) data type, i.e. "time", a running (unsegmented)
sequence of seconds (as floats with arbitrary resolutions).
The closest approximation to it (given all "external" constraints)
is what proposed in 3.4 i.e. 2D doublets of relative times
with respect to a reference.
We could as well call these 2D (with the *fixed* split at
integer/fraction, see discussion above) as the "time data type".
And the reference time, due to the foresight of the FITS fathers
(:-) smile but not ironic) is stored as an ASCII string in a
loosely-typed, arbitrary resolution header kwd, either as [M]JDREF
or DATEREF.
Which is *correctly* referred to as a "pseudo data type".
For me, ISO-8601 strings pertain correctly to header keywords (for
which all types are "pseudo"). If one thinks of a binary table
then I would never consider storing individual times as 19-char or
longer ISO-8601 strings, so for me it is not a TFORM but a TDISP,
a format for display !
However, even I would probably never use an ASCII table instead
of a binary table, ASCII tables are not ruled out, nor 20A
columns in binary tables are ruled out.
That is, even if I'd done perhaps differently, I could be
contented with the current wording of 3.1 and 3.4 (but for
the 'if and when' portions, see below).
WP> I think these "if and when" type statements have no place in a
WP> reference standards document. [...] I think the presence of these "if
WP> and when" statements in the paper serve no useful purpose and will
WP> lead to continued diversionary debate as we proceed through the formal
WP> approval process of this FITS paper, and therefore should be deleted.
The same applies to the other "if and when" at the end of section 3
(about 128-byte floats).
I tend to agree with Bill.
(although to be honest there have been cases, e.g. in the BINTABLE
paper, of features present in the paper and not part of the standard,
or which only later became part of the standard)
---> Precisely; and see my response to Bill's comments
WP> If one wants to propose a new datetime data format for FITS, then I
WP> think this should be done separately so as to not further delay the
WP> completion of this Time WCS paper.
The argument in favour of prompt publication (and standard
endorsement) of this paper is indeed strong.
However, to be realistic, one shall say that, if we have to define
a time data type ... either we do it now or we postpone it to a
far future.
As said above, I do not consider we need a "datetime" type (which is a
TDISP for me). We might need a "time" type (or "high resolution
time"), and that could simply be the 3.4 2D case with strict rules on
it, cutting out variants and possible choices.
---> Enough said
Let's look forward to a prompt freezing and submission of the paper !
--
------------------------------------------------------------------------
Lucio Chiappetti - INAF/IASF - via Bassini 15 - I-20133 Milano (Italy)
------------------------------------------------------------------------
Italian Research STILL at risk. La Ricerca italiana TUTTORA a rischio !
see http://sax.iasf-milano.inaf.it/~lucio/WWW/Opinions/nobrain3.html cfr
_______________________________________________
fitswcs mailing list
fitswcs at listmgr.cv.nrao.edu
http://listmgr.cv.nrao.edu/mailman/listinfo/fitswcs
----- End of forwarded message from Lucio Chiappetti -----
[cleardot]
-------------------------------------------------------------------------------------------------------------
Arnold H. Rots Chandra X-ray Science
Center
Smithsonian Astrophysical Observatory tel: +1 617 496 7701
60 Garden Street, MS 67 fax: +1 617 495
7356
Cambridge, MA 02138
arots at cfa.harvard.edu
USA http://hea-www.harvard.edu/~arots/
--------------------------------------------------------------------------------------------------------------
------- end of forwarded message -------
More information about the fitswcs
mailing list