[Pafgbt] save PAF cross-correlations rather than formed beam outputs
Karl Warnick
warnick at ee.byu.edu
Mon May 20 16:27:19 EDT 2013
One comment below..
Karl
On 5/20/2013 12:48 PM, Brian Jeffs wrote:
> Comments below.
>
> Brian
>
>
> On May 20, 2013, at 10:28 AM, Karl Warnick wrote:
>
>> I agree that correlations are important. Correlated outputs will
>> probably be the default mode for most PAF observations, and real time
>> beams would only be used when phase information is important, such as
>> for pulsar surveys (can pulsar searches be done with fine grained,
>> narrowband correlations?)
>>
>> I'm not sure whether the bandwidth issue is the real driver. Real
>> time beams also have to be formed in subbands and then either
>> processed for pulsar surveys or integrated for power spectra.
>> I don't think bandwidth is really what drives the difference between
>> a B engine and an X engine. It's true that if one only forms a few
>> PAF beams, then with a given disk storage data rate one could store
>> the beam outputs over finer frequency subbands than correlations, but
>> if fully sampled beams are formed, the data rate wouldn't be very
>> different between beam outputs and correlations with the same
>> integration time and frequency resolution. Am I missing something?
>
> For the same integration time and frequency resolution, roughly the
> same number of beams as PAF elements (fully sampled beams and Nyquist
> array sampling), ignoring the N log N complexity growth of an FFT,
> ignoring processor architecture constraints, and if you don't need
> time series data for downstream processing, then there is no relative
> advantage between an integrating B engine and an X engine as to data
> rate. There are however some very real practical limitations.
>
> If you have an ASKAP PAF you have many more elements than beams, so a
> B engine is more efficient. With our PAF, which has about the same
> number of beams as elements, the calculus changes.
>
> _All frequency channelization must be performed before the
> correlations (X engine) stage_, so if you need fine-grained frequency
> resolution you need a huge FFT to handle the 800 Msamp/s target sample
> rate of our back end. We don't have a processor architecture that can
> do that FFT size (due to N log N growth). You must do coarse FFTs
> first, then either real-time beamform or correlate. If you correlate
> (e.g. and then beamform) you can never get to the fine frequency
> resolution. If you real-time beamform, you can follow with another
> FFT stage to get fine frequency channels. You could then correlate,
> or integrate squared spectra, or send to pulsar processing as suites
> the application.
If we were correlating outputs of an N channel PAF over the full
bandwidth, wouldn't the cost of a fine FFT, while large, be negligible
in comparison in relation to the correlator? When I say fine grained,
narrowband correlations, I mean a fine FFT followed by X engine, with
the integration time very small. The time scale certainly can be no
smaller than N_FFT samples, but it could still be pretty small. That's
certainly possible in principle, with a large enough back end. I wonder
if there are applications that would be well served by this kind of
architecture, with a fine F engine followed by a correlator.
>
> For pulsar work I don't know what the frequency resolution
> requirements are, so I can't judge whether you can beamform after
> correlation. I don't think "fine grained, narrowband correlations"
> are possible since to be narrowband means correlations would
> necessarily be coarse grained in time.
>
>>
>> There is another motivation for beamformers in the broader PAF
>> community, and that is for synthesis arrays. The correlator for the
>> dish array is expensive, so one would only want to correlate as small
>> a number of beams from each PAF as possible. This raises a question -
>> could correlations from each PAF be used to get correlations of the
>> dish array somehow? Is there an efficient two-level correlator
>> architecture, with a PAF correlator for each dish, followed by the
>> synthesis array correlator? I suspect not, but I can't quite convince
>> myself. In any case, it seems that for single dish telescopes,
>> there's less motivation to use a beamformer back end instead of a
>> correlator.
>
> I agree with this. That is why ASKAP uses a real-time beamformer
> ahead of the correlator. Correlation across the dishes requires
> frequency channelized times series, not integrated data. I have not
> seen proposals for a two-level correlator. The first stage (PAF
> level) correlation removes all absolute time or phase reference
> information for one dish relative to another, which is the key signal
> component needed to form the 2nd level (dish to dish) correlation.
> The other reason for doing real-time beamforming at the PAF level is
> to reduce data transport requirements (many more elements than beams).
> Eigenbeams further reduce the data transport load. Integrated
> correlations would of course be even lower data rate, but I don't see
> how to get the cross dish visibilities from PAF correlations.
>
> For a single dish PAF the arguments are weaker for real-time
> beamforming unless actual time samples or superfine spectral
> resolution are needed.
>
>>
>> Finally, there is probably a bit of analysis one could to to show how
>> closely beams must be spaced in order for the information in the
>> beams to be equivalent to the correlations. It's essentially the
>> problem of recovering the matrix R from a set of inner products
>> w'*R*w for many vectors w. I suspect that if one forms HPBW/2 spaced
>> beams over the PAF FoV, the information in the beams is less than but
>> on the order of the information in R in some quantiable sense.
>> Information in the deep sidelobes is lost, but most of the large
>> eigenvalues of R represent sources in the field of view. With finer
>> beams, even just over the PAF field of view, all information even out
>> in deep sidelobes could well be contained in the beam outputs, but
>> that's a moot point, as one would not form that many beams in
>> practice. There's also the idea of eigenbeams proposed by Cornwell et
>> al., so that one can form very few PAF beams yet still have
>> information over the full field of view.
>
> I agree with this.
>
>
>>
>> Karl
>>
>>
>> On 5/20/2013 9:25 AM, Brian Jeffs wrote:
>>> Rick,
>>>
>>> I agree that you have much more flexibility to try different
>>> beamformer designs, detection algorithms, interference mitigation
>>> techniques, superresolution, calibration correction, etc. if you
>>> store and operate on the accumulated cross products (correlation
>>> matrices). However, you give up the ability to do fine resolution
>>> spectral processing. You are stuck with the coarseness of the
>>> correlator's frequency channelization. I don't know how problematic
>>> this is for some applications, such as pulsar searches, where fine
>>> spectral resolution may be needed.
>>>
>>> Brian
>>>
>>> On May 20, 2013, at 6:38 AM, Anish Roshi wrote:
>>>
>>>>
>>>> Yes indeed. We can form images with beams with different
>>>> optimization if the correlations are recorded.
>>>> Anish
>>>>
>>>>
>>>> On Sun, May 19, 2013 at 9:57 AM, Rick Fisher <rfisher at nrao.edu
>>>> <mailto:rfisher at nrao.edu>> wrote:
>>>>
>>>> Brian, Karl,
>>>>
>>>> In trying to understand the ASKAP data processing architecture, I'm
>>>> beginning to understand the fundamental importance of saving the
>>>> cross-products between array element outputs in our own PAF data
>>>> processing. In forming beams you throw away a lot of
>>>> information in the
>>>> array's field of view that can be recovered only by forming
>>>> many beams
>>>> with very close spacing (much closer than HPBW/2). This has
>>>> important
>>>> consequences for the sensitivity to point sources, as in the
>>>> search for
>>>> pulsars. Hence, I would suggest that the most important
>>>> archived outputs
>>>> from your signal processor are the cross-products rather than
>>>> formed
>>>> beams. For a given data storage volume, there's more
>>>> information in the
>>>> cross-products than in the formed beam outputs. In some
>>>> respects, the
>>>> "beam" concept is a holdover from a waveguide feed where
>>>> there's only one
>>>> output, and most of the information in the focal plane is
>>>> reflected back
>>>> into the sky.
>>>>
>>>> Rick
>>>> _______________________________________________
>>>> Pafgbt mailing list
>>>> Pafgbt at listmgr.cv.nrao.edu <mailto:Pafgbt at listmgr.cv.nrao.edu>
>>>> http://listmgr.cv.nrao.edu/mailman/listinfo/pafgbt
>>>>
>>>>
>>>> _______________________________________________
>>>> Pafgbt mailing list
>>>> Pafgbt at listmgr.cv.nrao.edu <mailto:Pafgbt at listmgr.cv.nrao.edu>
>>>> http://listmgr.cv.nrao.edu/mailman/listinfo/pafgbt
>>>
>>>
>>>
>>> _______________________________________________
>>> Pafgbt mailing list
>>> Pafgbt at listmgr.cv.nrao.edu
>>> http://listmgr.cv.nrao.edu/mailman/listinfo/pafgbt
>>
>> --
>> Karl F. Warnick
>> Department of Electrical and Computer Engineering
>> Brigham Young University
>> 459 Clyde Building
>> Provo, UT 84602
>> (801) 422-1732
>>
>>
>>
>>
>>
>> _______________________________________________
>> Pafgbt mailing list
>> Pafgbt at listmgr.cv.nrao.edu <mailto:Pafgbt at listmgr.cv.nrao.edu>
>> http://listmgr.cv.nrao.edu/mailman/listinfo/pafgbt
>
--
Karl F. Warnick
Department of Electrical and Computer Engineering
Brigham Young University
459 Clyde Building
Provo, UT 84602
(801) 422-1732
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/pafgbt/attachments/20130520/a52149f2/attachment.html>
More information about the Pafgbt
mailing list