[evla-sw-discuss] BigEndian flaw in the BDFReader

[Keith Cummings]

FYI

On Friday I discovered that the original BDFReader has a flaw that 
causes ~0.4% of the float range to be read incorrectly.  The code 
attempts to read the float value from the binary sections of the BDF 
file into a Java float as BigEndian.  It then checks the BDFHeader to 
see if the byte-order is actually LittleEndian and if it is (which is 
always is), it reverses the bytes and creates a new float.

The problem arises because there are "illegal" bit patterns for a float 
in Java.  Per the Java 6 JavaDocs, "If the argument is any value in the 
range |0x7f800001| through |0x7fffffff| or in the range |0xff800001| 
through |0xffffffff|, the result is a NaN."  Reversing the byte-order of 
Float.NAN results in 6.9055E-41 which is the number the BDFReader 
returned for any floats in the "illegal" ranges.  If the bytes had be 
read as LittleEndian in the first place this would never happen and the 
code now does this.

As far as TelCal is concerned (which is the only user of the BDFReader 
that I'm aware of), this bug existed until June 15, when I deployed a 
new version of TelCal (1.5.14) that used the new bdfReader2 code.

More analysis would be needed to see how much of our pre-June15 data 
mapped to the "illegal" ranges.

It should be noted that the June 15 deployment of TelCal also adjusted 
the flagged integration range so that flags apply to integrations before 
and after the integration the flag occurred during, without crossing 
subscan boundaries.  This flagging change plus the BigEndian fix should 
result in higher quality data since June 15, all else being equal.

-Keith
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/evla-sw-discuss/attachments/20100628/7d175ae8/attachment.html>