[wfc] CHECKSUM Proposal

[William Pence]

The objectors to the CHECKSUM proposal (see below) have argued that random
transcription errors are so rare that there is no need for an additional
layer of file validity checking at the FITS file level.  This overlooks what
I regard as one of the most important uses of the CHECKSUM keywords: The
CHECKSUM keywords provide a simple mechanism for putting a 'validity stamp'
or 'seal of approval' on the FITS data files that are retrieved from large
public archives like the HEASARC, MAST, or NRAO.  By verifying that the
CHECKSUM in the FITS file is still correct, the end user can be reasonably
assured that the local file is identical to the file in the archive, and
that it has not been modified (either deliberately, or inadvertently) by
subsequent data analysis software.  For this purpose, it is actually better
if most data analysis software ignores the CHECKSUM keywords and does not
automatically update or delete them.  Then the fact that the file fails the
checksum test will clearly indicate that the current file is not the same as
the file that was retrieved from the archive.  Failing the CHECKSUM test
does not mean the file is not a valid FITS file.  It only means that the
file is no longer the same as when the CHECKSUM was originaly computed.

-Bill Pence

Robert Hanisch wrote:
> 
> In the past I have expressed my reservations about the checksum proposal,
> i.e., whether it is really necessary.  Computer systems have so much
> inherent error checking built in to them that I wonder if adding our own
> additional check really adds anything.  I also worry a bit about people
> making manual changes to FITS files, which invalidate the checksum but in
> fact do not affect the integrity of the file.  

Arne Henden wrote:
> 
> I have fairly strong feelings *against* checksums in fits files.
> After handling several hundred thousand CCD frames, the only ones
> I found corrupted were *really* corrupted and were obvious.
> Files that are transmitted over sockets or networks can take
> advantage of their retransmittal capability to handle transfer errors;
> all hard drives are pretty robust these days, and if errors
> are important, can be RAIDed.  So error handling should be at
> the system level, not the internal file level.


-- 
____________________________________________________________________
Dr. William Pence                          pence at tetra.gsfc.nasa.gov
NASA/GSFC Code 662         HEASARC         +1-301-286-4599 (voice)     
Greenbelt MD 20771                         +1-301-286-1684 (fax)