[fitsbits] Start of the Public Comment Period on 2 Image Distortion Conventions

[David W Hogg]

(If you respond to this, please CC me, because I am *not* on FITSBITS.)

Here are a few cents of mine; I will ask Dustin Lang to follow up as
well.  I warn you that I am writing this *without* having read the
documents, but I figure if I wait until I read the documents, I will
never get my comments in in time.

-----
We have been using SIP at http://astrometry.net/ for a few years now,
and have working C and Python code that manipulates, creates, writes,
and reads SIP.  Overall, we like SIP, for the main reason:

(0) SIP is very clear and simple, to read, implement, adjust, and analyze.

We adopted SIP over TNX at the time for two reasons:

(1) SIP was in image coordinates, TNX was in sky coordinates (at the
time, this may have changed), and we expect *most* (not all) fixed
distortions to be fixed in camera and not sky coordinates.

(2) SIP made use of the usual FITS keywords, and did not require a
string keyword manipulator to extract distortion coefficients, as TNX
did at the start (don't know if it does still).

My main problems or issues with SIP are as follows:

(3) The inverse transformation is just a separate transformation and
there is no restriction that the inverse *actually be* the inverse or
any approximation to it.  Indeed, it can't be an exact inverse.
However, computers are smart and fast, so the inverse *could* have
been encoded as the Newton's method inverse of the forward, and only
one transformation would be stored in the header at all!  I consider
this an issue for a number of reasons:

(3.1) is it wrong to invert the transformation by iterating the
forward transformation?  If so, how do we prevent that?

(3.2) there are many ways to approximate the reverse transformation
with a polynomial, and different investigators will make different
choices (least squares on the calibration stars?  least squares on
arbitrary control points? use errors how?).  This is not standardized,
I suspect, in the standard.  If it *were* standardized, would we force
SIP readers to check that?

(4) The restriction to 9th order may not seem restrictive, but if you
know your camera has repeatable radial distortions, and you have
millions of images, then you can easily fit radial terms beyond 9th
order.  Up to 9, these can be encoded as a SIP with many terms set to
exactly zero.  After 9, no dice.

(5) The standard does not prevent an investigator from inputting a
distortion map that is multi-valued, folded, and non-invertible, in
forward or in reverse.  In fact, once you get to high enough order, it
is very hard to write a SIP header that does *not* have these problems
somewhere in the "focal plane".

There is one over-arching problem with all of these standards, which
perhaps deserves addressing:

(6) If you read the definitional WCS papers, the authors were clearly
thinking of WCS to describe the mapping of a processed, output map.
They were not thinking of using WCS to describe obtained, natural
images.  They were imagining that the investigator deals with all the
natural images in some idiosyncratic way, produces a distortion-free
map, and specifies the properties of that map using the WCS
convention.  That is *not* the way WCS is used in most applications,
and leads to some of the compromises encountered in TNX and SIP.

Finally, one last irrelevant comment:

(7) One extreme WCS standard that we *could* put forward, and which
*would* work in all circumstances is the following:

(7.1) A list of ra, dec values, and a matched list of x,y values.
Interpolate as you wish!

(7.2) This standard would allow distortions of any kind, automatically
includes the forwards and reverse transformations, does not need to
piggy-back on the existing WCS TAN (or any other) undistorted
standard, permits distortions of any order or character, permits the
investigator to make the distortion finer in some parts of the image
than others, etc. etc.  It is not fully specified, of course, but a
full specification would not be hard to write.

(7.3) Implemented standards are better than non-implemented standards,
so for now we still prefer SIP.

David
-- 
David W. Hogg - associate professor, NYU - http://cosmo.nyu.edu/hogg/