[mmaimcal]Imaging Project Book Chapter
Mark Holdaway
mholdawa at nrao.edu
Fri Feb 22 17:26:19 EST 2002
Dear Fellows,
I've been working on a new version of the Imaging Project Book chapter.
This is very different from the old one, which was pretty insufficient.
Furthermore, it walks closely to the business of computing,
but with a different emphasis.
I'm not yet done with this work, but the stuff is sufficiently different
that I thought I would put this draft out for high level comments about
the approach and all.
Take care,
-Mark
-------------- next part --------------
Imaging Requirements for the ALMA
13.1 Observing Modes
Imaging requirements for ALMA is a complicated topic which is partially
covered in the Calibration, Computing, and Configuration chapters.
We will try to minimize the overlap with these other areas, but will
have to repeat some information for clarity.
ALMA's imaging is complicated by the diversity of observing modes
which ALMA is designed to perform, the physical limitations of the
antennas as we push to extremely high frequencies, and the extraodinarilly
high sensitivity of the array. A low sensitivity array may suffer from
the physical limitations of the antennas, but a high sensitivity array
such as ALMA will have the SNR to actually fix some of these problems.
Also, ALMA's imaging must be understood in unprecedented detail
because of the requirements of dynamic scheduling and pipeline
imaging. In the past, these demands have been glossed over, and we
must look at the imaging requirements with a new seriousness.
Consider the subsystems of conceptual bodies of Observing Modes,
Proposal Submission Materials (or tools), Data Simulator, Current
Environmental Data, Dynamic Scheduler, On-Line Computing, and the
Imaging Pipeline. We are mainly concerned with certain interactions
between these bodies, and observing modes will be fundamental in
framing these interactions, as we need to be able to reduce data from
all supported observing modes. ALMA will have a number of observing
modes:
interferometric
single field interferometric
multi-field interferometric
fast switching
pointing measurement
total power
single pointing frequency switching
position switching
subreflector nutation
on-the-fly (OTF)
... and combinations of the above modes
pointing measurement
interferometric plus total power (mosaicing)
interferometric plus dedicated total power data
homogeneous array mosaicing (total power data taken
with the full interferometric array)
OTF mosaicing
interferometric plus ACA plus dedicated total power data
holography
interferometric
single dish (interferometry with reference reciever)
VLBI
Some of these observing modes require nothing of the imaging
algorithms. For example, data taken in pointing measurement mode will
be treated through calibration, and will usually not require any
special attention from the imaging algorithms.
Additionally, there are a number of data collection modes which might
distinguish several observations with the same observing mode:
continuum, spectral line, polarization, pulsar gating, phased array,
etc. The data collection modes correspond broadly with correlator
modes, as the correlator is the primary backend device for filtering
and manipulating which data are to be collected.
Finally, there are a number of reduction methods. There should be one
or more reduction methods for each reasonable (ie, desirable)
combination of data collection mode and observing mode. There will be
a number of levels of support for reduction methods: unsupported (ie,
the algorithm has not been implemented in ALMA software), beta support
(ie, the algorithm works, but may need scientific supervision to
proceed), and pipeline support (the algorithm is understood
sufficiently well to be automated).
Let us say the combination of observing mode, data collection mode,
and reduction method corresponds to an "observing intent". Two
different observations may have identical observing modes and data
collection modes, but if one observation is a long, thermal
noise-limited, detection observation while the other is a dynamic
range-limited observation of a bright source, they present very
different requirements to the ALMA scheduling system and reduction
system, and may have different reduction methods associated
with them.
Two observations with different "observing intent" might still be
imaged with the same reduction method. Conversely, two observations
with the same "observing intent" might be imaged with different
algorithms.
13.2 High Level Statement of ALMA Imaging Requirements
A top-level statement of the imaging requirements for ALMA are:
- to borrow, optimize, or generate, imaging algorithms for each
reasonable "observing intent", and to elevate the robustness and
understanding of each such algorithm to the level of pipeline
support in a manner which reflects both the demand for that
algorithm and the difficulty of the implementation of that
algorithm. It is understood that the set of "reasonable observing
intents" may evolve with time.
- to identify interactions between the imaging algorithms'
performance and other aspects of the ALMA system, including the
proposal process, the simulator, the scheduler, the site
environment, and the imaging pipeline.
- to quantify inherent limitations on the various imaging algorithms,
and the relationships between the site's environmental data and the
imaging algorithm's success. For example, there will be a
correlation between wind and antenna pointing, and also
correlations between antenna pointing and image quality for
different observing modes. The scheduler will need to have a clear
statement of the relationship between environmental data and
potential image quality to make good choices with sheduling.
13.3 Relationships Between Observing Modes (Observing Intents) and
Other Subsystems
The subsystems of Proposal Submission, Simulator, Environmental Data,
Scheduler, On-Line System, and Pipeline rely upon the observing modes
for specific content and instruction, and their purpose is to fulfill
the goals of the observing intent. Most of the interactions of these
subsystems is outside the scope of imaging, but the interactions of
these subsystems with the observing modes is central to imaging.
The current capabilities of the ALMA will be clearly identified
and advertised in the Proposal Submission subsystem. We need to state
which observing modes and intents are supported by the simulator, the
on-line system, the automatic scheduling system, the off-line
reduction software, and the automated pipeline imaging. Furthermore,
the Proposal Submission system must know the various parameters which
are required for each observing mode or intent. Many parameters will
be common to nearly all observing modes, but some will be more
specific. For example, pointing errors are usually of little concern
for single pointing obervations, but may become an important
limitation for multi-field observations, and a pointing error limit
parameter would need to be included for mosaicing modes.
As alluded to above, the simulator should understand the relevant
details of each supported observing mode or intent. As the simulator
matures, it will also be able to simulate the most likely limiting
errors for a given observing mode or intent.
One of the most important areas of interactions between the observing
intents and the rest of the ALMA computing system is the estimation of
quality of an observation which is about to be performed, given the
current environmental site data. This is the calculation which the
dynamic scheduler must perform constantly to determine the optimal use
of the array. This area of interactions can be summarized by a list
of quasi-analytical, numerical, or purely empiracle relationships
between envirmonmental conditions and the final image quality. For
example, a very simple relationship might be high SNR mosaic
observations above a certain frequency are never performed during
daytime hours (ie, when solar radiation and strong winds will cause
adverse surface, voltage pattern, and pointing effects, thereby
decreasing the quality of the mosaiced images).
However, such a simple rule (no daytime mosaics at high frequency)
could be fine tuned a great deal. For example, a detection experiment
may not require very high image quality, and may tolerate some day
time conditions. Or more quantitatively, we might find that when the
solar flux is less than some amount and there is no solar shadowing of
antennas by other antennas, we can make moderate quality images up to
a certain observing frequency. Finally, we may have some sort of
correction algorithm which improves the image quality further, or
pushes the maximum recommended observing frequency higher for a given
set of environmental conditions (see Image-Plane Effects below).
So, one of our tasks is to compile the set of all such relevant rules
which relate the environmental conditions and the observing mode to an
estimated image quality. These rules must be compiled for all
observing intents to be considered by the dynamic scheduler. Initially,
many of these rules will be rough, ad hoc rules, based on intuitive
understanding of the behavior of the array and the imaging algorithms.
However, experience on the array, especially experience gleaned from
analysis of test data which has been correlated with environmental
conditions, and extensive simulations of the various observing modes
and either environmental conditions or antenna errors derived from the
environmental conditions, will be very useful in formulating improved
rules. Obviously, this is an area of exploration which must begin now,
but will be ongoing through the life of ALMA.
Of course, there is an obvious relationship between the observing
intents and the off-line reduction system or the Pipeline system.
Each mode and intent needs to be supported with off-line reduction
software, and will eventually be supported in an automated manner
through the imaging pipeline. The observing intent needs to be
present, either explicitly by some category name, or implicitly
embedded in information about the required image quality, the
source SNR, and the correlator setup, at the time of observation
so the imaging pipeline can sensibly image the data.
Likewise, each observing mode needs to be archivable. In addition to
the usually archived information, we will also need to archive any
information available on the intent of the observation. Data from the
archive will at times be reconstituted and passed through the imaging
pipeline, so there needs to be some sort of persistence of observing
intent, yet also some flexibility so that the pipeline recognizes
a superior algorithm might be used which was not available at the
time of the archived observations.
13.4 Observing Intents
The following attempts to be a complete list of observing intents for
ALMA. Each broad observing category is followed by a number of lines,
each with an exclusive switch such as [ continuum | spectral line ].
A large number of possible observing intents follows from the outer
product of all the exclusive switches. Many of the observing intents
under a given observing category might be imaged with the same
algroithms. Some intents will require specialized algorithms either
for higher efficiency (when low SNR justifies an approximation) or for
high imaging accuracy (when high SNR justifies a more stringent
approach and various errors must be handled explicitly). At this
time, no effort has been made to prioritize the support for the
various intents. Intents required for ALMA commissioning would
be top priority, followed by common observing modes, intents
which can be split off and imaged more efficiently, and finally
the most demanding intents which require new imaging algorithms.
- total power point detection:
[ Beam Switched | Frequency Switched | Position Switched ]
[ continuum | spectral line ]
[ total intensity | polarization ]
- total power imaging:
[ OTF | point & shoot ]
[ Beam Switched | Frequency Switched | No Switching ]
[ continuum | spectral line ]
[ total intensity | polarization ]
[ high SNR | low SNR]
For high SNR total power imaging, effects such as pointing errors and
beam irregularities might be accounted for.
- compact single field
For compact single field imaging, a single region in the primary
beam (presumably the center) is being sampled. Hence, image-plane
effects such as pointing errors and beam errors become negligible or
manifest themselves as visibility-based errors which can be
corrected by self-calibration.
[ continuum | spectral line ]
[ total intensity | polarization ]
[ high SNR | moderate SNR | low SNR]
Low SNR observations require Fourier inversion, but no deconvolution.
Moderate SNR observations require deconvolution.
High SNR observations require iterative rounds of deconvolution
and self-calibration.
- non-compact single field
While a single field is sufficient to image the target source, the
object is spread over the primary beam. High SNR observations may
require attention to image-plane effects. Polarization observations
may require attention to the polarization beam, another image-plane
effect.
[ continuum | spectral line ]
[ total intensity | polarization ]
[ high SNR | moderate SNR | low SNR]
Low SNR observations require Fourier inversion, but no deconvolution.
Moderate SNR observations require deconvolution.
High SNR observations require iterative rounds of deconvolution
and treatment of image-plane effects and/or self-calibration.
- multi-field observations of many compact sources
Interferometric observations alone can accurately represent the
source structure, or the extended source structure is being ignored,
so total power measurements are not required. High SNR observations
may require attention to image-plane effects.
Separately deconvolved images of each field, combined into a
single mosaic image after deconvolution, will reduce the magnitude
of many image-plane effects.
Holography observations are a special case here (a multi-field
observation of a single compact source). Since there is only one
source, the image-plane effects reduce to visibility-based effects
and can be used to solve for the image-plane errors.
[ point & shoot | OTF ]
[ continuum | spectral line ]
[ total intensity | polarization ]
[ high SNR | moderate SNR | low SNR]
Low SNR observations require Fourier inversion, but no deconvolution.
Moderate SNR observations require deconvolution (presumably prior to
combination into a single mosaiced image).
High SNR observations require iterative rounds of deconvolution
and treatment of image-plane effects and/or self-calibration.
- multi-field observations of extended sources
Interferometer and total power data must be combined to produce a
single image. In some cases, the total power may be collected with
the all dishes and at the same time as the interferometric data
(homogeneous array). As nutators will be present on only four
antennas, continuum observations probably cannot be performed with
the homogeneous array concept. The four antennas with nutators will
form a dedicated total power subarray, which will likely operate
independently of the rest of the array, observing the target source
at a different time than the main array. The ACA, another means of
measuring short spacings, is not currently budgeted, but is listed
for completeness.
[ dedicated total power | homogeneous array |
ACA & dedicated total power]
[ point & shoot | OTF ]
[ continuum | spectral line ]
[ total intensity | polarization ]
[ high SNR | moderate SNR | low SNR]
Low SNR observations can use linear mosaicing with a single approximate
deconvolution. Moderate SNR observations require full deconvolution.
High SNR observations require treatment of image-plane effects,
dealing with both the interferometric and total power data.
13.5 What Imaging Algorithms Do We Need?
COMPLETE THIS
Total Power Imaging
Single Field Imaging
-reduction of single field interferometer data. Many different
algorithms exist to accomplish this, and it is pretty well
understood. There will be more work in this area when ALMA comes
on line, but we can't realy predict what this work will be.
Much of the single field interferometer data will result in thermal
noise-limited images. These images will have signal to noise
ratios (SNR) ranging from tens to one to hundreds to one. If this
SNR is less than the ratio of point spread function (PSF) peak to
maximum sidelobe ratio, the image may not require deconvolution.
One issue to be aware of in undeconvolved images which are limited
by thermal noise is that the potential SNR will likely be much
greater on shorter baselines or at low resolution, but that the PSF
sidelobe level may not be similarly improved at that resolution.
In this case, the undeconvolved images would be limited on the
large scales by failure to deconvolve. This is an important reason
for multi-scale PSF optimization with respect to both weighting
and array design.
-weighting (also applies to multi-field): The Fourier plane coverage
for the various configurations of ALMA is exceptionally good,
resulting in a nicely shaped PSF with sidelobes of only a few
percent. Furthermore, a simple reweighting of the data which gives
an optimally Gaussian PSF and sidelobes of a few tenths of a percent
will only cost on the order of 10\% of the array's sensitivity
(Boone, 2002). This indicates that many observations will not
require any deconvolution at all. We should embark upon a study of
such reweighted but undeconvolved images to determine the nature of
their errors and to know under what conditions deconvolution will
improve the images (for example, if the source size is equal to or
greater than the recirpocal of the shortest measured spacing,
deconvolution could greatly increase the recovered flux, generally
imroving the image quality).
-fast switching (also applies to multi-field); while the imaging
system is not primarily concerned with fast switching (it is a
calibration technique), the fast switching can result in some
baseline-dependent decorrelation leading to imaging sources which
appear to be a bit more resolved than they actually are. A simple
correction algorithm is suggested by the fast switching method.
The phase detection carried out on the calibrator will provide us
with the statistics of the residual phase errors on each baseline.
From this, we can estimate the decorrelation each baseline
experiences on the target source and divide the visibility
amplitude by the decorrelation factor. If this is done, the noise
will increase on the longest baselines, so we should also
explicitly downweight the visibilities by a commensurate amount,
leaving the net visibility amplitude times the weight unchanged.
However, the beam's weights are also changed, which will match the
beam to the manner in which point sources are incorrectly resolved
by the baseline-dependent decorrelation. Hence, baseline-dependent
decorrelation can be handled by simply adjusting the weights for
the calculation of the PSF, and not adjusting the weights or
visibilities that go into the dirty image calculation at all. Upon
deconvolution with the reweighted PSF, point sources will be
recovered where true point sources are located. This is a topic
for more work.
Multi-Field Imaging and Other Wide Field Issues
-linear mosaicing: Just as moderate dynamic range single-field
observations can sometimes forego deconvolution, multi-field
observations can also make some shortcuts. The design of the most
compact ALMA configuration (ie, the ``mosaicing configuration'') is
driven mainly by the need for high surface brightness sensitivity,
and the antennas are so close together that there is not a great
deal of room for sidelobe minimization and beam shaping, but the
compact configuration's PSF will still be pretty good. A first
order mosaic image would be the linear combination of the dirty
images. This option, and it's implications for total power, need
further exploration; the homogeneous array's dirty images suffer
from the zero-spacing problem (ie, negative bowels masking extended
structure), and require deconvolution to reconstruct the large scale
emission. As mosaicing will be one of the most CPU-intensive
imaging pipeline activities, a great easing of the Pipeline's
computational requirements would result if a significant fraction of
observations could utilize linear mosaicing without deconvolution.
Another option for a quick mosaic, which takes more CPU time, but
which has been demonstrated to work for homogeneous array data, is
to deconvolve the entire linear dirty mosaic image for the effects
of a single mean PSF (Cornwell, Holdaway, and Uson, 1994; AIPS++
User's Manual, 2002). In reality, each pointing will have a
slightly different Fourier plane coverage and a different PSF.
However, if the mosaic observation was made wisely, each pointing
will have a very similar PSF, and the main effects of the PSF on the
image can be removed by deconvolving the mean or effective PSF from
the linear mosaic of the dirty images. This deconvolution strategy
can be applied to objects with potential SNR of several hundred to
one, and it too represents a significant savings over a full
non-linear mosaic. Several different deconvolution methods could
be used at this point, including MEM, clean, multi-scale clean,
and, if complete effective Fourier plane coverage is achieved
(as it always in in the ALMA compact configuration), Wiener
filtering or linear deconvolution.
-non-linear mosaicing: Moderate and high SNR mosaics will require a
more exact treatment of the differences in (u,v) coverage among each
pointing. Effective non-linear mosaicing algorithms have been
implemented in SDE, Miriad, Gildas, and AIP++. Deconvolution
methods in non-linear mosaicing algorithms include MEM, maximum
emptiness, clean, and multi-scale clean. The algorithmic
limitations of non-linear mosaicing are generally beyond the SNR
expected of most ALMA observations. However, even with a bright
source and low thermal noise in the ALMA system (ie, a high
potential SNR), there are a host of image plane effects which could
otherwise limit the image quality of the mosaic images. The following
sections detail some of these effects and propose a possible solution
to move beyond these limitations.
-wide field (image plane) errors that ALMA must be concerned with
COMPLETE THIS
* pointing
COMPLETE THIS
* surface errors and other miscelaneous voltage pattern errors
* illumination allignment; The tapered illumination of the primary
dish by the feed will not be perfectly centered on the primary.
The shift in the aperture illumination corresponds to a phase gradient
in the sky voltage pattern. If not corrected, this voltage pattern
error could severely limit mosaics at frequencies up to several
hundred GHz. However, the shift in aperture illumination is
almost totally equivalent to a shift in the (u,v) coordinates
which rotates as the parallactic angle. An excellent fix to this
problem has been demonstrated (Holdaway, 2001), and it is handled
in the calibration chapter.
*polarization: On-axis polarization issues are treated in the
calibration chapter. Wide-field polarization imaging (ie,
non-compact single field or multi-field observations of extended
sources) will be affected by the impurity of the voltage patterns.
The off axis feeds of two supposedly orthogonal result in
non-orthogonal voltage patterns, which is equivalent to an
instrumental polarization which changes with position in the beam.
Like the instrumental polarization, this polarization beam rotates
with parallactic angle. If the voltage patterns are known, and we
can generate a good model of the total intensity image, we can
calculate the spurious polarization signal for each measured
visibility and subtract it from the data visibilities. A simple
image-plane version of this problem has been implemented for
snapshot observation on the VLA (Cotton, 1995). More on this sort
of correction algorithm is found in the section on image-plane
errors below.
-an approach to image-plane errors in AIPS++
At high frequencies, ALMA mosaic images will be limited by
pointing errors and surface errors. We expect that there
will be many cases where the source will be bright enough
and ALMA will be sensitive enough to solve for these errors
and push beyond this dynamic range limit into a new regime
of high SNR mosaicing. However, it is unclear how much success
we might expect from this direction of inquiry.
Image plane errors, such as pointing errors, destroy the nice
conceptual break between visibility data and images. A pointing
error is a calibration parameter, yet a correction for a pointing
error cannot be applied to the visibilities for a non-compact
source. However, given a model brightness distribution and the
form of the image-plane error, it is straightforward to compute
the visibility. Pointing errors and beam errors are essentially
direction dependent gains. A general solution to the problem of
direction-dependent gains is quite demanding.
The design of the AIPS++ system has the concept of such
direction-dependent gains built in. Furthermore, the possibility
exists of fixing the image-plane errors for just the brightest
sources, which, if uncorrected, would scatter the most flux,
thereby limiting the dynamic range and obscuring fainter emission.
Hence, the proposed strategy for image-plane problems is:
1) identify the bright problem sources which would limit the
imaging,
2) perform a cycle of imaging, collecting "flux components" down to
the level where we are beginning to become corrputed by
scattered spurious emission,
3) perform a sort of direction-dependent self-cal in these
directions, comparing the data visibilities to the model
visibilities to solve for the parameters image plane errors,
4) calculate the model visibilities implied by the source
distribution revealed so far, augmented by the solved
image-plane error parameters,
5) subtract the model visibilities from the data visibilities
6) continue with step 2), performing the next cycle of imaging
of weaker emission.
This proposed algorithm is new and uncertain territory, and should
be prototyped as soon as possible.
-wide field errors that we probably don't have to worry about:
* focus errors; Focus errors will introduce a concentric phase
error pattern in the illumination voltage, which results in
phase errors in the sky voltage pattern. If two antennas have
the same focus error, this effect cancels as that baseline's
primary beam is formed by multiplying one sky voltage pattern
by the complex conjugate of the other's voltage pattern.
Focus errors may help confuse other sorts of errors, such
as pointing errors, but numerical simulations indicate
focus errors will not be a problem for mosaicing with ALMA.
* total intensity beam asymmetries from offset feeds; as the feeds
get further and further off axis, the beams are expected to become
deformed and asymmetric. Deformed or asymmetric beams will rotate
on the sky with parallactic angle and can be handled in the
mosaicing process with a small increase in computation, as long as
they remain constant with time. Calculations based on James
Lamb's aperture illumination voltage calculations indicate that
the 30~GHz beam will be symmetric to a very high degree, less than
a fraction of a percent.
* non-coplanar baselines will not be a problem. Low frequency
observations of wide fields at high resolution results in the
situation where the synthesized beam varies over the primary beam.
This becomes an issue when the quantity \lambda B_{max} / D^2
becomes larger than 1. For 14~km baselines, 1~cm wavelength, and
12~m antennas, this diagnostic is 1.0, indicating that for the
highest resolution, lowest frequency ALMA observations, the
non-coplanar array will be a minor problem for complicated fields,
and it will not be any sort of problem on shorter baselines or at
higher frequencies.
13.5 The Simulator
COMPLETE THIS!
13.6 What Image Quality-Environmental Condition Relationships
Do We Need?
As stated previously, this will be a growing web of logic which will
start out very sketchy and will be filled in through simulations and
experience on the ALMA array. These are the relationships which will
drive the dynamic scheduler, and the array's efficiency will be
closely tied to the utility and accuracy of these relations. However,
we seek to document a starting point with a series of guesses. These
relations should be redocumented here as they are adopted in the
dynamic scheduling process.
The first two relations do not really belong here, as they are not
directly related to imaging, but more to calibration. However,
in the interest of colocating all scientific equations which indicate
some aspect of observational success in relation to the
environmental data, we include them here as well.
COMPLETE THIS
noise -- opacity -- elevation, trade against (u,v) coverage
phase stability
pointing -- wind
pointing -- anomalous refraction
pointing -- thermal
pointing -- errors in pointing determination frozen in until
next calibration cycle
beam shape -- wind
beam shape -- thermal
what else?
These image quality - environmental data relationships are a key part
of the dynamic scheduler, but we still need an algorithm for using
them to choose the optimal observing project with time. Other
non-environmental factors, such as calibration overhead for a project
switch and the need to complete projects which have been partially
observed, will also enter in to the project selection process.
References
AIPS++ User's Manual, 2002.
Boone, 2002.
Cornwell, Holdaway, and Uson, 1994.
Cotton, 1995.
Holdaway, 2001, Illumination Offset Memo
Lamb, James, 2001, private communication (James provided an aperture
voltage pattern for the 30 GHz system, and I calculated the sky voltage
patterns and made inferences).
More information about the mmaimcal
mailing list