[alma-config] Three possible metrics

[Ed Fomalont]

Hi Dave et al,

    Your metric of >40% u-v coverage over the uv-plane for an
acceptable synthesis array is a good first principle.  I believe that
this simple metric could be expanded to be more indicative of other
qualities of the u-v coverage for an experiment.  Two additional
metrics are suggest and these are related to the smoothness of
coverage.

    One metric is related to the detailed lumpiness of the u-v
coverage.  For example, consider three adjacent u-v cells with weights
0, 1, and 5.  (Usually the weighting is related to the amount of
integration time of data within the cell.)  Although two of the three
cells have non-zero weights, I believe that the difference in weight
between 1 and 5 can produce larger side-lobe problems in the dirty
beam than the weighting difference between 0 and 1.  You could put
this in another way.  Even if all 100% of the u-v cells are occupied,
significant extra sidelobes would arise because of the different
weighting amongst the cells.  Of course, using 'uniform' weighting
where all cells have weight 0 or 1 elliminates this problem, but at a
cost of signal to noise.  The importance of this degradation depends
on the details of the array and the goals of the experiment.  Other,
more intermediate weighting types (Robustness in the lingo of AIPS),
can and should be used.

    The second criterion is large-scale lumpiness.  For example, if the
60% of the cells with no u-v data were randomly distributed, this would
be much better than if the 60% loss of cells were in a wedge or in a
ring in the u-v plane.

    Can these two smoothness criteria be quantified?  I think so in
the following way.

1.  The u-v cell size is somewhat arbitrary, but a reasonable cell
size is related to the field of view.  For a primary beam FWHP beam
size of N (minutes of arc), a reasonable image size is 3N, which gives
a field of view out to the first nulls of the primary beam.
Occasionally, strong sources can be detected further out but the 3N
field of view seems reasonable.  This gives a cell size DW = 1250/N
(wavelengths) where N is the FWHP in arcmin.  The number of cells over
the u-v coverage depends on the size of the array, and could vary from
200 to 10000.

2.  Metric 1-The percentage of cells filled: By gridding the data for
the specific observation, the percentage of cells which are occupied
can be easily calculated.  The total area in the u-v plane of
consideration should be elliptical with the semi-major axes and
position angle given by the extremes of the u-v coverage.

3.  Metric 2-The smooth u-v coverage: The general smoothness of the
u-v coverage can be obtained in a variety of ways.  I would suggest
'gridding' the u-v plane into large cells, say 10x10 original cells
for the smaller configurations and 100x100 for the larger
configurations, and determining the data weight in each of these big
cells.  A reasonable rule of thumb is to have about sixteen of these
big cells in each of the two directions of the u-v coverage.  By data
weight I mean the integration time of the data in each of these cells,
but other weighting schemes, as suggested above, could be used.  (This
type of gridding is similar to that using the adverb UVBOX in IMAGR in
AIPS.)

    For a 'good' array, this average u-v coverage should be a smoothly
decreasing function of distance from zero spacing, the smoother the
better.  Fit the distribution of these u-v densities to the best
elliptical Gaussian (maybe something else is better like a density
related to the inverse distance from the center).  The rms deviation
of the average u-v distribution to this best fitting Gaussian is a
measure of the overall smoothness of the actual u-v coverage.  Any
significant large-scale 'bumps' will increase metric 2 and will
definitely cause large scale sidelobes in the dirty beam.  For
example, missing wedges in the u-v plane will clearly affect this
metric.  As a side note, in order to decrease the effect of the
synthetic aperture edge and the near-in sidelobes that this edge
produces, a Gaussian taper can be added to the u-v coverage before
making this calculation.

   A normalized metric which measures this overall smoothness of the
u-v coverage could be:

    W(i) is the weight of data in the ith big cells
    E(i) is the weight of the best fitting elliptical Gaussian to
         the distribution of W(i) over the u-v plane.
         (any tapering of the data included before this gridding)
    NT   is the total weight of data.
    M2   is Metric 2

             M2 = SUM(i){ [(W(i)-E(i)]**2 } / NT*NT

The metric would be small for relatively uniform coverage, but large if
there are missing wedges or rings in the u-v coverage, or big holes
of any type.  I think it is normalized in a reasonable manner.

    Of course, all array u-v coverage will have a central hole with a
size of the diameter of the array telescope.  I don't know what the
thinking is for the ALMA array.  Is it expected that the hole will be
filled using on of many techniques, all of which have their problems?
In any case, this hole can or can not be included in the Metric 2,
whatever is felt most appropriate.

4.  Metric 3-The detailed lumpiness:  For each of the big u-v cells,
which were used in the above smoothness calculation, calculate the
roughness of uv-coverage.  Something like the following algorithm
could work:

    W(i) is the weight of data in the ith big cells
    w(i,j) is the weight of the jth u-v cell in the
         ith big cell
    n    is the number of little cells in each big cell
    NT   is the total weight of data.
    M3   is Metric 3

       M3 = SUM(i){ SUM(j) {[W(i)/n - w(i,j)]**2 } } / NT*NT

5.  How are these metrics related to the properties of the dirty beam?
Metric 3 and metric 1 are probably related to the overall power in
small-scale sidelobes in the dirty beam, with the maximum sidelobe
related to some combination of Metric 1 and 3.

    Lower level (usually) large-scale sidelobes, either a wedge
distortion due to missing wedge or rolling waves caused by more
compact regions of missing spacings, are probably related to Metric 2.
These distortions may be relatively small in peak magnitude but can
contain more 'integrated' power than the larger, smaller-scale
sidelobes.  The sidelobes close to the main beam response are
associated with the aperture edge and these can be lessened by
tapering the u-v coverage at the longer spacings.  This is not related
to any of the three metrics.
 

SUMMARY:

    Three metrics are suggested for the evaluation of array performance:

Metric 1: The percentage of cells with no data.  
Metric 2: The overall smoothness of the cell population 
Metric 3: The detailed lumpiness of the cell population

    Since two of these metrics are only at the thought-experiment
stage, I am not sure how valuable they will be.  My guess is that
Metric 2 is the most important, then Metric 1 and then Metric 3 in the
evaluation of an experiment.  Perhaps, the evaluation of something
like Metric 2 and 3 could be done from the dirty beam directly,
instead of the u-v coverage.  Maybe so, but these metrics are an
attempt to measure the amount of power in large-scale and small-scale
features on the dirty beam, so it seems better to stay in the (u-v)
plane which is a direct measure of these frequencies.  But, only
simulations will tell which, if any, are simple indications of a
relatively complicated measurement of array performance.

    Thanks,  Ed