[alma-config] thoughts on simulation metrics

[John Conway]

Hi,

 Bryans list is a useful summary of the metrics that have been suggested.  
Due to the time constraints  I think we only have a  chance to implement
one or two before the  Grenoble meeting.

At the last telecon I was asked to make a suggestion on metrics-
but I have had less time to to think about it than I expected
-so I apologize for not producing anything up to now.

I quite liked Dave Woody's suggestion - which is in Bryans list 
(see below). I think it gives us the quantitative information we want.
However I would slightly  modify it to make the axes more intiative
as I describe below.


> 
>       4. Dave Woody has made a suggestion:
> 
>          What about doing a simple linear fit of the (diff-map)^2 to
>          A + B*(original simulation image)^2 ?
>          1/sqrt(B) would be interpreted as the fidelity, i.e., the 
>          errors in the map that are proportional to the image. 
>          1/sqrt(A) would be the "off-source" dynamic range.
>          This fit should not be computationally time consuming or 
>          difficult to code.
> 

A similar suggestion was made by Stephane a while ago.

I would suggest modifying, so it works with binned data rather than 
a scatter plot (so the full dynamic range curve can be displayed for each
simulation)  and also use slightly more intuatitive 
axes (i.e rms vales rather than  square values).

In the modification take bins on the x axis equally spaced in 
Log I_model (perhaps  bins covering a factor of two in model 
intensity)

For a given intensity bin - look at all pixel locations in the ERROR image 
having a intensity value in the MODEL which lies in  the intensity range
of the bin - for these  pixel locations calculate the rms of the pixels in
the ERROR  image. Plot the Log of this quantity as the y value.

This will produce a plot of Log(rms error) versus Log(I_model).
It will be interesting to see the shape of this plot, it should
be displayed for each model/array_size/array_type CLEAN simulation.
The curves might  be charactered by a straight line at large Log I
implying a  single  on-source dynamic range - or more likely the 
shape will be a curve implying different on-source dynmic range 
as a function of model intensity  - plus it will have  a saturation
at low Log I  from which we can estimate the off-source errors and the 
off source dynamic range.  In any case  if sufficinetly close to linear
its useful to 

rms error  =  B I_model + A.

as in Daves original suggestion where now 
B is the on-source dynmamic range and A the size of off source errors (to
be divided by the peak value of the model to get 'off-source' dynamic
range as conventionally defined by radio astronomers). If it is very
significantly curved then a higher order polynomial can be fitted.

Also useful to plot would be Log(rms error/I_mod) vs Log I OR
Log(I_Mod/rms error) versus Log(I) to give the dynamic range 
directly as a function of Log I.  

        John.