[Pafgbt] save PAF cross-correlations rather than formed beam outputs

Tue May 21 11:37:44 EDT 2013

Anish,

Yes, this is the way we do it to find the signal vector for calibration, with an eigen decomposition.  I think we were the first to propose that approach.  We have an improved method using a generalized eigenvector decomposition which also uses a dark sky noise covariance matrix estimate contemporary with the calibration observation.  It is in:

Elmer, M.; Jeffs, B.D.; Warnick, Karl F. ; Fisher, J.R. ; Norrod, "R.D.Beamformer Design Methods for Radio Astronomical Phased Array Feeds," Antennas and Propagation, IEEE Transactions on, Volume: 60, Issue: 2, Part: 2, Digital Object Identifier: 10.1109/TAP.2011.2173143, 2012 , Page(s): 903- 914.

Brian

On May 21, 2013, at 7:14 AM, Anish Roshi wrote:

Hi Rick,

A clarification (see below) --

On Mon, May 20, 2013 at 4:02 PM, Anish Roshi <anish.roshi at gmail.com<mailto:anish.roshi at gmail.com>> wrote:

Anish can correct me if I'm wrong, but I think he said that the ASKAP default plan is to cross-correlate all array PAF elements with all other PAF elements across all synthesis array antennas, which avoids the need to form voltage beams.

No they correlate between all PAF elements in an antenna to get the weights for 30 beams. Then they will correlate between the 30 beams for synthesis array. Eigen beam forming is not what they are going for in the first place.

Getting the weights for the beam has two steps -- (1) to measure the noise correlation matrix using all the PAF elements
on a blank sky and (2) measuring the `signal vector'. They take the `signal vector' (say S) as the complex voltages obtained by cross correlating elements of the PAF in one antenna with a feed (it is a horn at present; but I think one element of PAF can also be used) in a second antenna.

As I was discussing with Doug Hayman about the measurement of the signal vector -- I think S will not give the signal vector (Brain may be interested in this since I heard he will at ASKAP for a year). This is because of mutual coupling in PAF elements. One should form a matrix SS^H and diagonalize the matrix. The `signal vector' should then be the eigen vector corresponding to the dominant eigen value.

 Anish

This is more expensive in signal processing power than forming modestly spaced voltage beams, but it retains all of the information in the focal plane at full sensitivity.
Forming "eigen beams" may be a compromise between these to approaches.

Yes that is right -- at least that is my understanding.

Anish

Good discussion regardless of the final outcome.

Rick

On Mon, 20 May 2013, Brian Jeffs wrote:

Comments below.
Brian

On May 20, 2013, at 10:28 AM, Karl Warnick wrote:

      I agree that correlations are important. Correlated outputs will
      probably be the default mode for most PAF observations, and real
      time beams would only be used when phase information is
      important, such as for pulsar surveys (can pulsar searches be
      done with fine grained, narrowband correlations?)

      I'm not sure whether the bandwidth issue is the real driver.
      Real time beams also have to be formed in subbands and then
      either processed for pulsar surveys or integrated for power
      spectra.

      I don't think bandwidth is really what drives the difference
      between a B engine and an X engine. It's true that if one only
      forms a few PAF beams, then with a given disk storage data rate
      one could store the beam outputs over finer frequency subbands
      than correlations, but if fully sampled beams are formed, the
      data rate wouldn't be very different between beam outputs and
      correlations with the same integration time and frequency
      resolution. Am I missing something?

For the same integration time and frequency resolution, roughly the same
number of beams as PAF elements (fully sampled beams and Nyquist array
sampling), ignoring the N log N complexity growth of an FFT,  ignoring
processor architecture constraints, and if you don't need time series data
for downstream processing, then there is no relative advantage between an
integrating B engine and an X engine as to data rate.  There are however
some very real practical limitations.

If you have an ASKAP PAF you have many more elements than beams, so a B
engine is more efficient.  With our PAF, which has about the same number of
beams as elements, the calculus changes.

All frequency channelization must be performed before the correlations (X
engine) stage, so if you need fine-grained frequency resolution you need a
huge FFT to handle the 800 Msamp/s target sample rate of our back end.  We
don't have a processor architecture that can do that FFT size (due to N log
N growth).  You must do coarse FFTs first, then either real-time beamform or
correlate.  If you correlate (e.g. and then beamform) you can never get to
the fine frequency resolution.  If you real-time beamform, you can follow
with another FFT stage to get fine frequency channels.  You could then
correlate, or integrate squared spectra, or send to pulsar processing as
suites the application.

For pulsar work I don't know what the frequency resolution requirements are,
so I can't judge whether you can beamform after correlation.  I don't think
"fine grained, narrowband correlations" are possible since to be narrowband
means correlations would necessarily be coarse grained in time.

      There is another motivation for beamformers in the broader PAF
      community, and that is for synthesis arrays. The correlator for
      the dish array is expensive, so one would only want to correlate
      as small a number of beams from each PAF as possible. This
      raises a question - could correlations from each PAF be used to
      get correlations of the dish array somehow? Is there an
      efficient two-level correlator architecture, with a PAF
      correlator for each dish, followed by the synthesis array
      correlator? I suspect not, but I can't quite convince myself. In
      any case, it seems that for single dish telescopes, there's less
      motivation to use a beamformer back end instead of a correlator.

I agree with this.  That is why ASKAP uses a real-time beamformer ahead of
the correlator.  Correlation across the dishes requires frequency
channelized times series, not integrated data.  I have not seen proposals
for a two-level correlator.  The first stage (PAF level) correlation removes
all absolute time or phase reference information for one dish relative to
another, which is the key signal component needed to form the 2nd level
(dish to dish) correlation.  The other reason for doing real-time
beamforming at the PAF level is to reduce data transport requirements (many
more elements than beams).  Eigenbeams further reduce the data transport
load.  Integrated correlations would of course be even lower data rate, but
I don't see how to get the cross dish visibilities from PAF correlations.

For a single dish PAF the arguments are weaker for real-time beamforming
unless actual time samples or superfine spectral resolution are needed.

      Finally, there is probably a bit of analysis one could to to
      show how closely beams must be spaced in order for the
      information in the beams to be equivalent to the correlations.
      It's essentially the problem of recovering the matrix R from a
      set of inner products w'*R*w for many vectors w. I suspect that
      if one forms HPBW/2 spaced beams over the PAF FoV, the
      information in the beams is less than but on the order of the
      information in R in some quantiable sense. Information in the
      deep sidelobes is lost, but most of the large eigenvalues of R
      represent sources in the field of view. With finer beams, even
      just over the PAF field of view, all information even out in
      deep sidelobes could well be contained in the beam outputs, but
      that's a moot point, as one would not form that many beams in
      practice. There's also the idea of eigenbeams proposed by
      Cornwell et al., so that one can form very few PAF beams yet
      still have information over the full field of view.

I agree with this.

      Karl

      On 5/20/2013 9:25 AM, Brian Jeffs wrote:
      Rick,
I agree that you have much more flexibility to try different
beamformer designs, detection algorithms, interference
mitigation techniques, superresolution, calibration correction,
etc. if you store and operate on the accumulated cross products
(correlation matrices).  However, you give up the ability to do
fine resolution spectral processing.  You are stuck with the
coarseness of the correlator's frequency channelization.  I
don't know how problematic this is for some applications, such
as pulsar searches, where fine spectral resolution may be
needed.

Brian

On May 20, 2013, at 6:38 AM, Anish Roshi wrote:

Yes indeed. We can form images with beams with different
optimization if the correlations are recorded.
Anish

On Sun, May 19, 2013 at 9:57 AM, Rick Fisher
<rfisher at nrao.edu<mailto:rfisher at nrao.edu>> wrote:
      Brian, Karl,

      In trying to understand the ASKAP data
      processing architecture, I'm
      beginning to understand the fundamental
      importance of saving the
      cross-products between array element outputs
      in our own PAF data
      processing.  In forming beams you throw away a
      lot of information in the
      array's field of view that can be recovered
      only by forming many beams
      with very close spacing (much closer than
      HPBW/2).  This has important
      consequences for the sensitivity to point
      sources, as in the search for
      pulsars.  Hence, I would suggest that the most
      important archived outputs
      from your signal processor are the
      cross-products rather than formed
      beams.  For a given data storage volume,
      there's more information in the
      cross-products than in the formed beam
      outputs.  In some respects, the
      "beam" concept is a holdover from a waveguide
      feed where there's only one
      output, and most of the information in the
      focal plane is reflected back
      into the sky.

      Rick
      _______________________________________________
      Pafgbt mailing list
      Pafgbt at listmgr.cv.nrao.edu<mailto:Pafgbt at listmgr.cv.nrao.edu>
      http://listmgr.cv.nrao.edu/mailman/listinfo/pafgbt

_______________________________________________
Pafgbt mailing list
Pafgbt at listmgr.cv.nrao.edu<mailto:Pafgbt at listmgr.cv.nrao.edu>
http://listmgr.cv.nrao.edu/mailman/listinfo/pafgbt

_______________________________________________
Pafgbt mailing list
Pafgbt at listmgr.cv.nrao.edu<mailto:Pafgbt at listmgr.cv.nrao.edu>
http://listmgr.cv.nrao.edu/mailman/listinfo/pafgbt

_______________________________________________
Pafgbt mailing list
Pafgbt at listmgr.cv.nrao.edu<mailto:Pafgbt at listmgr.cv.nrao.edu>
http://listmgr.cv.nrao.edu/mailman/listinfo/pafgbt

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/pafgbt/attachments/20130521/c91a0f6c/attachment.html>