WFC: Year-2000 issue (III)

[Arnold Rots]

This file contains a proposal for how to revise the definition of the
DATExxxx keywords to provide a 4-digit year.  It was developed as a
distillation of discussion on the sci.astro.fits newsgroup, electronic
mail interchanges, and discussion with Preben Grosbol. It has been
submitted to the FITS committees by Peter Bunclark of the Royal
Greenwich Observatory.

Attached is a newly amended version of the Bunclark proposal that
can be described as following the EFC endorsement with the
following exceptions, additions, and deletions:

1. The use of the UTC time scale is mandatory and implied for data
   that originated since 1972, unless an alternative, valid time
   scale is specified through the TIMESYS keyword.

2. Time scales that will be initially recognized are: UTC, TAI, ET,
   TT, TDT, TDB, TCG, TCB.

3. Support for use of the "Z" qualifier is deleted since it would be
   either superfluous or create an inconsistency.

4. The default interpretation of all DATExxxx keywords shall use the
   Gregorian Calendar.


For a convenient summary of time scales, see:
   http://tycho.usno.navy.mil/systime.html

I have one question left: the EFC did not want the appendix to be part
of the proposal.  My recollection is that there seemed to be consensus
in the WFC that it _should_ be made part of the proposal.
Am I right?  In that case we should add:

5. The Appendix should be considered part of the proposal.

To summarize our concerns that led to the changes:
a. UTC is not defined before 1972.
b. FITS files should, internally, not mix time scales; there are files
   around that use TT, TDB, TAI.
c. Not all code needs to know about all time scales; in at least 95%
   of the cases it would not hurt if UTC were assumed.

I would suggest that WFC forward this proposal to the IAU FITS Working
Group.

  - Arnold

--------------------------------%<----------------------------------------
 
                  Precise re-definition of DATE-OBS Keyword
                         encompassing the millennium
 
                         Peter Bunclark 1996 Nov 19
                      Amended: Arnold Rots 1997 May 23
 
                             1) Introduction
The original DATExxxx keywords, including in particular DATE-OBS,  have
several shortcomings which make it desirable to alter the definition:
1.1) The year is expressed in only 2-digits;  currently, digitized astronomical
      data spans more than a century, and further, the implied most-significant
      two digits of the year will change from 19 to 20 shortly.
1.2) The timescale of DATExxxx is not defined.
1.3) The relation of DATE-OBS to the start, middle or end of an observation
     is not defined.
1.4) The order of day, month, year is least-significant first, so lists
     of dates cannot be sorted simply on the ASCII collating sequence.
 
                               2) Scope
There are three main issues addressed;
2.1) the format of date strings to be used in any DATExxxx keyword.
2.2) the future of the DATE-OBS keyword itself
2.3) the specification of the time scales (time systems) used

                    3) The Date-String format Proposal
 
3.1) A DATExxxx field in the old format of 'DD/MM/YY' will explicitly refer
   to a year 1900-1999.  The very few instances of digitized nineteenth-century
   plates represented as FITS files must be handled as special cases.
 
3.2) The new recommended format is a restricted subset of
   ISO-8601, being one of two options:
   a) 'ccyy-mm-dd'
   b) 'ccyy-mm-ddThh:mm:ss[.sss...]'
 
   In the short form, there may not be any additional terminator/separator
   (such as T).  In the long form, there must be a 'T' time designator
   between the date and the time.
   The decimal point character is an ascii full-stop.
   The number of decimal places in the `seconds' field may be arbitrarily 
   long, up to the FITS header-card limitations.
 
3.3) Only fully-specified date or date/time strings are permitted. No fields
   may be defaulted, no leading zeroes omitted.  The decimal part of
   the seconds field is optional.
 
                    4) Use of the DATE-OBS keyword
 
4.1) The name of the keyword shall remain DATE-OBS.
 
4.2) It is suggested that the DATE-OBS refer to the start of an observation.
   This relationship should be clarified in the comment field.
 
4.3) The time scale or time system shall be specified by means of the
   keyword TIMESYS.

4.4) The default interpretation of all DATExxxx keywords shall use the
   Gregorian Calendar.

                    5) Use of the TIMESYS keyword

5.1) Use of the keyword TIMESYS is recommended.  It will set the time
   system for all time related keywords and data in the HDU.
   Initially, officially allowed values are:
      UTC (Coordinated Universal Time; defined since 1972)
      TAI (International Atomic Time)
      ET  (Ephemeris Time, the predecessor of TT; valid until 1976)
      TT  (Terrestrial Time, the IAU standard time scale since 1976;
           continuous with ET and synchronous with TAI)
      TDT (Terrestrial Dynamical Time; =TT)
      TDB (Barycentric Dynamical Time)
      TCG (Geocentric Coordinate Time)
      TCB (Barycentric Coordinate Time).
   For reference, see:
      Explanatory Supplement to the Astronomical Almanac, P. K. Seidelmann,
      ed., University Science Books, 1992, ISBN 0-935702-68-7 

5.2) Times will be deemed to be as measured at the detector (or in practical
   cases, at the observatory) unless a different location is implied by the
   time system specified (such as is the case for TCG, TDB, TCB).

5.3) The preferred value is the IAU standard, "TT", which is considered
   equivalent to "TDT" and "ET", though "ET" should not be used for data
   taken after 1976.

5.4) If the TIMESYS keyword is absent or has an unrecognized value,
   the value "UTC" will be assumed for dates since 1972.  Note that, by
   implication, the time scale for pre-1972 data may be undefined.

 
                          6) Examples
Four legal representations of the date of the first formal draft of this
proposal are possible:
 
DATE-OBS= '14/10/96'           / Original format, means 1996 Oct 14.

TIMESYS = 'UTC     '
DATE-OBS= '1996-10-14'         / Date of start of observation in UTC.

DATE-OBS= '1996-10-14'         / Date of start of observation, also in UTC.

TIMESYS = 'TT      '
DATE-OBS= '1996-11-19T13:38:36.307' / Date and time of start of obs. in TT.
 
                          7) Transition
 
FITS readers must continue to interpret the old format, as a twentieth
century date, forever.  Readers should be altered as soon as possible
to cope with the new format.  In order to give adequate time for the
major package writers to adopt their systems, FITS writers should commence
writing the new format between 1998-01-01T00:00:00 and 2000-01-01T00:00:00.
 
                8) Appendix: Relevant ISO 8601 paragraphs
 
5.2.1.1 Complete representation
When the application clearly identifies the need for an expression only
of a calendar date, then the complete representation shall be a single
numeric data element comprising eight digits, where [CCYY] represents a 
calendar year, [MM] the ordinal number of a calendar month within the
calendar year, and [DD] the ordinal number of a day within the calendar month.
 
Basic format: CCYYMMDD
Example:      19850412
Extended format: CCYY-MM-DD
Example:         1985-04-12
 
5.3.1.3 Representation of decimal fractions
If necessary for a particular application a decimal fraction of hour, minute
or second may be included.  If a decimal fraction is included, lower order
components (if and) shall be omitted, and the decimal fraction shall be
divided from the integer par by the decimal sign specified in ISO 31-0:
ie the comma [,] or full stop [.].  Of these, the comma is the preferred sign.
If the magnitude of the number is less than unity, the decimal sign shall
be preceded by a zero (see ISO 31-0).
The number of digits in the decimal fraction shall be determined by the
interchange parties, dependent upon the application.  The format shall be
[hhmmss,s], [hhmm,m] or [hh,h] as appropriate (hour minute second, hour
minute and hour, respectively), with as many digits as necessary following
the decimal sign. If the extended format is required, separators may be
included in the decimal representation when the complete representation is
used, or when it is reduced by omission of [ss,s]
....
Example: 23:20:50,5
....
 
5.4.1 Complete Representation
The components of an instant of time shall be written in the following
sequence:
a) For calendar dates:
year-month-day-time designator-hour-minute-second
b) For ordinal dates:
   year-day-time designator-hour-minute-second
c) For dates identified by week and day numbers:
   year-week designator-week-day-time designator-hour-minute-second
 
The character [T] shall be used as time designator to indicate the start of
the representation of date time of day in combined date and time of day
expressions. The hyphen [-] and the colon [:] shall be used, in accordance
with 4.4, as separators within the date and time of the day expressions
respectively, when required.  When any of the date or time components are
omitted, the time designator shall always precede the remaining time of day
components.
....