[daip] [!7296]: AIPS - OMFIT and strange chi2 values

[Michael Bietenholz]

Michael Bietenholz updated #7296
--------------------------------

OMFIT and strange chi2 values
-----------------------------

           Ticket ID: 7296
                 URL: https://help.nrao.edu/staff/index.php?/Tickets/Ticket/View/7296
           Full Name: Michael Bietenholz
               Email: mbieten at yorku.ca
             Creator: User
          Department: AIPS Data Reduction
       Staff (Owner): -- Unassigned --
                Type: Issue
              Status: Open
            Priority: Default
                 SLA: NRAO E2E
      Template Group: Default
             Created: 22 September 2015 08:35 PM
             Updated: 23 September 2015 12:50 PM
                 Due: 24 September 2015 10:24 PM (1d 9h 34m)
      Resolution Due: 30 September 2015 08:35 PM (7d 7h 44m)



Here's some more diagnostics for this - something strange is going on with OMFIT and the Chi2, which may have to do with the order of the random parameters (the previously fixed factor-of-two issue was with the reported uncertainties, this time its with the reported Chi2)

I made two small data sets, '1JY.WTMOD' and '1JY SN.wtmod', which have all visibilities = (1,0) Jy [exactly, no noise] and wts of 1.0 (Attached).  I'm fitting a model with zero flux density and no free parameters (zero.omfit input file also attached).  So the fit is trivial and Chi2 = sum-of-squared-residuals should be just Nvis.  

The first issue is that OMFIT does *not* get "Real Chi2"=Nvis for these data sets.  The "Ketan Chi2" seems now to be the correct one!
Perhaps when you reorganized the code, you removed whatever scaling for Chi2 that Ketan put in, and Ketan's Chi2 *is* now the actual Chi2.

The second issue is that OMFIT gets different values of "Real reduced Chi2" for these two data sets, even though they are very similar.  The only structural difference I can see is that the random parameters are in a different order between the two data sets:

1JY:        UU-L-SIN   .... GATEID  CORR-ID  ANTENNA1  ANTENNA2
1JY SN:   UU-L-SIN  .... GATEID  ANTENNA1  ANTENNA2  CORR-ID


With the default inputs for OMFIT and the attached zero.omfit input I get this:

"1JY"    Nvis="Ketan Chi2"=234.00; DOF=468; "Real Chi2" = 936.0; "Real reduced Chi2" = 1.000

So here, the "Ketan Chi2" is actually correct, and the "Real Chi2" is wrong, being 4x the correct value.  Here, NOISE(1)=0 and NOISE(1)=1 give the same result, as I would expect it to.

But now on the second one with NOISE(1)=0

"1JY SN": Nvis="Ketan Chi2"=154.00; DOF=308; "Real Chi2" = 598.0; "Real reduced Chi2" = 1.9415584

So "Ketan Chi2" still has the expected value, but "Real Chi2" is now no longer 4x the correct value, but rather 3.88x
the correct value (??)  Now if I change NOISE(1) to 1.0, then "Real Chi2" changes to 299, and "Real reduced Chi2 to 0.9708.

------------------------------------------------------
Staff CP:  https://help.nrao.edu/staff
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1jy.fits
Type: image/fits
Size: 210240 bytes
Desc: not available
URL: <http://listmgr.nrao.edu/pipermail/daip/attachments/20150923/fc59c892/attachment-0002.fits>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 1jy_sn.fits
Type: image/fits
Size: 241920 bytes
Desc: not available
URL: <http://listmgr.nrao.edu/pipermail/daip/attachments/20150923/fc59c892/attachment-0003.fits>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: zero.omfit
Type: application/octet-stream
Size: 284 bytes
Desc: not available
URL: <http://listmgr.nrao.edu/pipermail/daip/attachments/20150923/fc59c892/attachment-0001.obj>