[Difx-users] Constraint on the FFT resolution to correlate the wide-band (e.g., 2 GHz) data

[Ludwig Schwardt]

That's an interesting problem...

Let N be the number of wide-band channels, Fs the sampling rate in Hz and
Fr the fringe rate in seconds per second.

The usual duration of an FFT in seconds = 2 * N / Fs
The fringe rate in ns / s = 1e9 * Fr
The sampling time in ns = 1e9 / Fs

So the delta fractional sample delay becomes (2 * N / Fs) * (1e9 * Fr) /
(1e9 / Fs) = 2 * N * Fr, which is independent of the total bandwidth and/or
sampling rate.

If you want to keep this delta below say 0.1 (thereby ensuring 99.8%
efficiency), at the stated worst-case fringe rate of 1 microsec/s, this
suggests that you cannot have more than 50,000 channels in your wide-band
FFT, regardless of bandwidth.

Best regards,

Ludwig




On 24 August 2016 at 10:56, Lupin Lin <lupin at asiaa.sinica.edu.tw> wrote:

> Problem:
>
> In a simple test to correlate with the 2 GHz simulated data (4-sec
> duration in this example) of the flux in 2 Jy for CARMA (SEFD = 5), SMA
> (SEFD =10) and SMTO (SEFD =20), a strange trend can be found from the
> obtained result as shown in “HresFFT_2048MHz.pdf”.
>
> In the correlation obtained for CARMA-SMA and SMA-SMTO, an obvious
> coherence lost following the increasing of the frequency can be seen on the
> figure.
> But this trend does not appear on the correlation for CARMA-SMTO.
> Such a problem may be related to the simulator (e.g., the computation with
> the earth rotation since this problem occurs on the correlation to the long
> baseline).
> What’s the origin and how to fix it?
>
> ———————————————————————————————————————————————————
> Adam’s response:
>
> If you have a very large FFT and a high fringe rate, then the assumption
> that the fractional sample error remains constant for the duration of the
> FFT becomes incorrect.  This means that the frequency-dependent correction
> that is applied is also not exactly right, with the error being zero at the
> reference band edge and maximal at the other band edge.
>
> One can figure it out: work out the duration of an FFT in seconds,
> multiply that by the fringe rate in ns/s, and divide the result by the
> sampling time in ns.   If that number is large then you're starting to get
> substantial decorrelation at the far end of the band.
>
>
> In the high-resolution FFT plot for correlation (e.g., HresFFT_2048MHz.pdf)
>
> :
> The "FFTSpecRes = 0.00390625” (e.g., the duration for FFT is 2.56x10^{-4}
> sec) was determined in the .v2d file, so the pointer to examine the
> fractional sample error for 3 baselines are:
> CARMA-SMA: 0.61
> CARMA-SMTO: 0.10
> SMA-SMTO: 0.71
>
> With a delta fractional sample delay of 0.1 samples, at the high end of
> the band there will be sin(0.1)/0.1 ~ 99.8% correlation still.  But with
> 0.71, you have only 91.8% correlation.  That is not so far from what the
> plots seem to show. So I suspect if you reduce the spectral resolution by
> at least a factor of 8 you'll see matters improve greatly.
>
> For normal continuum observations I would expect that a channel width of
> ~0.5 MHz would suffice.  That leads to an FFT length of 2 microseconds.
> With a geometric rate of up to 1 microsec/s (maximum due to Earth
> rotation), then you have a delta geometric delay of 2e-12 seconds from one
> FFT to the next.  Since the sampling time for a 2 GHz band is 2.5e-10s,
> you're then looking at ~0.01 sample shifts per FFT, which is back within
> the range of negligible impact.
>
> It is good to remember that the wider the input band to DiFX, the coarser
> the best attainable spectral resolution without running into this problem.
> With a 2 GHz wide input band, you can't go much better than 62.5 kHz
> channels (at least on 8,000 km baselines) without accepting some
> decorrelation.  Whereas with a more typical band, like 128 MHz, you can go
> 16x narrower.  That is one of several moderately annoying limitations with
> very wide band inputs in DiFX (the others being mostly logistical, relating
> to chunk sizes, timekeeping etc).
>
> ———————————————————————————————————————————————————
> Examination on the problem:
>
> Here I put the revised correlated results here (LresFFT_2048M_Hz.pdf).
>
> To generate the new figure, I only changed "FFTSpecRes = 0.5”, which is
> 100x larger than the previous setting, and corresponding to the duration of
> 2x10^{-6} sec.
> Then the previous trend, “a clear declination in the amplitude (obtained
> S/N) following the increasing frequency”, has disappeared in the
> correlation for the baseline of CARMA-SMA and SMA-SMTO.
>
> ———————————————————————————————————————————————————
> Extensive Problem:
>
> One more question is about the zoom-band mode for the correlation.
> For example, we applied the zoom-band mode to examine the correlated
> result specified in a specific range (e.g., 1008-1040 MHz of the center
> band).
> For such a case, we saw the similar trend as we correlate the entire
> full-band (2 GHz) data with the precise “FFTSepcres”.
> (It means that S/N obtained at 4-36 MHz > it obtained at 1008-1040 MHz >
> it obtained at 2012-2044 MHz).
> But in the zoom-band mode, the bandwidth is now 32 MHz, not 2048 MHz.
> Or the mechanism to apply the zoom-band mode in correlation is still
> similar to correlate the full-band data, so I cannot set the FFT resolution
> too high.
>
> ———————————————————————————————————————————————————
> Adam’s response:
>
> As the name implies, zoom bands "zoom" in on the recorded bands.  All the
> standard signal processing (including the fractional sample correction) is
> done on the recorded band first.  So a zoom band just picks out a segment
> of the frequency spectrum of a recorded band, including any artifacts that
> have been introduced in the signal processing up to that point.  So zoom
> bands are subject to the same limitations regarding fractional sample
> correction as their parent wide bands.
>
> Getting around this would require adding a new filtering capability
> upstream of the standard station-based processing.  It would certainly be
> possible, and would be nice additional flexibility, but would be quite a
> bit of work.
>
>
> _______________________________________________
> Difx-users mailing list
> Difx-users at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20160824/a458c4c6/attachment-0001.html>