[wfc] CHECKSUM Proposal

[Rob Seaman]

Arne says:
 
> After reading the proposal, the concept and proposed implementation
> of the checksum is reasonable.  My objection is to the necessity of
> checksums at all.
 
Sounds fair.  Perhaps we should simply argue the necessity of checksums
from the established fact that more than one major FITS sites deems
them so.  Since millions of FITS files already contain checksums that
are compliant with the proposal, the question is whether it is more
useful and less complex to describe this - perfectly legal - FITS
usage within the standard or separately outside the standard.
 
> Certainly file integrity can be guaranteed better with external
> controls, along with possible correction as well as error detection.
 
External controls (RAID or what have you) will quite happily transcribe
errors, not just pristine pixels.  We need mechanisms that will travel
with the files from creation through to end users.  Those mechanisms
can be high security - and cost - like digital signatures, or low cost -
and moderate security - like checksums.

> If absolute integrity is important, the user should go back to the
> original source.

This will be a rather expensive option in the LSST era :-)
 
> Most image processing packages like iraf and Mira insert comments
> whenever the file is modified under most circumstances.

There are worlds of caution in the word "most".  Unless we start
working on a FITS standard for comments, these will never provide
more than anecdotal protection - and never in a machine readable
format.

A lack of comments is also pretty slim evidence for a lack of changes.
 
> If the proposal is accepted, then I would suggest a third keyword
> that indicates the type of checksumming used since there are several
> varieties and the authors suggest that theirs may be superceded in
> the future.

It is precisely the nature of the 1's complement algorithm that
allows it to be embedded within a particular HDU.  It is unlikely
that another algorithm will be adopted that has this property.
Other checksums, CRCs, message digests or digital signatures will
most likely be implemented using a separate FITS extension - perhaps
in a binary table, perhaps as a new extension type.  These will most
naturally provide only whole file protection, rather than the per-HDU
protection provided by the 1's complement ASCII encoded checksum.

In short, other types of checksums will extend the protection offered
by the algorithm expressed in the proposal - not replace it.

The original version of the proposal included a CHECKVER keyword.
After some discussion, we decided that this was unnecessary unless
and until a second per-HDU algorithm was identified.  Any checksum
contained within a separate extension will most likely be expressed
using a separate set of keywords and other data structures.

Rob Seaman