[Difx-users] complex support

[Adam Deller]

Glad to hear it!

On Thu, 11 Mar 2021 at 09:58, Jayce Dowell <jdowell at unm.edu> wrote:

> Hi Chris,
>
> After some digging I was able to get to the bottom of this and I am now
> getting results that I am happy with.  It was a combination of switching
> over to trunk and fixing a bad assumption in my VDIF conversion that
> everyone represents four-bit complex integers in the same way.
>
> Jayce
> --
> On 2/4/21 3:08 PM, Phillips, Chris (CASS, Marsfield) wrote:
> >    [EXTERNAL]
> >
> > Hi Jayce
> >
> > I'm working the data from the Long Wavelength Array and I am trying to
> see if we can move from our homegrown correlator to DiFX.  The two modes I
> need to support are a LWA-only mode with homogeneous data and an "eLWA"
> mode with heterogeneous data that combines the LWA stations with the VLA
> 4-band system.  For the LWA-only mode I have 4+4-bit complex voltage data
> from two 19.6 MHz wide spectral windows.  I am currently converting these
> data from the LWA format into two VDIF files, one per window, for
> correlation with DiFX.  My .input file describes the data as:
> >>
> >> DATA FORMAT:        INTERLACEDVDIF/0:1
> >> QUANTISATION BITS:  4
> >> DATA FRAME SIZE:    7872
> >> DATA SAMPLING:      COMPLEX_DSB
> >> DATA SOURCE:        FILE
> >>
> >
> > I have never personally tested complex_dsb data with interlaced vdif. I
> have no idea if there could be any “interaction” with the VDIF interleaving
> and the specific data type (I would have thought not, but you never know).
> >
> > The issue could also be with interpreting the interlaced VDIF, not the
> complex double sideband.
> >
> > Once you have tested using trunk, not the released version, I would try
> and use some of the vdifio utilities to try and convert the interlaced VDIF
> into plain VDIF - either extracting out one thread, or merging into
> multichannel single thread VDIF. You would obviously need to update the
> DIFX .input (and or .v2d) file and vexfile to match the new setup.
> >
> > I would also be using the m5access tools to do things like make the
> autocorrelation (both before and after fiddling with the data layout).
> Check DIFX gives the same auto shape as you get with m5spec.  If you use a
> modulated noise source, folding the data (m5fold) on the period of the
> noise cal can be very informative also.  Also look at the headers and check
> that the # frames/sec etc are what you expect (before and after merging
> threads etc). If you decide it is the interlacing is the issue, I suspect
> you will need to discuss with Walter.
> >
> >
> >> I am less sure about DiFX supporting our eLWA mode.  For this we have
> the same 4+4-bit complex voltage data from the LWA stations but with a
> bandwidth of 9.8 MHz.  We are combining this with 4-bit real voltages from
> the VLA with 8 MHz of bandwidth and a slightly different frequency setup.
> We currently process these data by shifting the center frequency of the LWA
> data to match that of the VLA, running two F-engines with different
> channelization to arrive at the same channel width for both data sources,
> and then cross correlate what overlaps.  Is this mode something that DiFX
> supports?
> >>
> >
> > Can you represent this data as VDIF? If you you can probably handle the
> different setup using zoom bands. However if this can be done will really
> depend on the EXACT setup. Basically you need to be able to channelise the
> two setups with the same final frequency resolution (ie different FFT sizes
> will be needed, with the ratio of the FFT size matching the ratio of the
> bandwidth difference. But the sky frequency of the frequency points out of
> the FFT need to match exactly (ie even if you could achieve the same
> frequency resolution, if the fine channels are offset by 1/3 of a channel
> you will have problems (if that offset is a simple integer ratio, you can
> just use a higher frequency resolution)
> >
> > I hope this makes sense.
> >
> > Cheers
> > Chris
> >
>
>

-- 
!=============================================================!
A/Prof. Adam Deller
ARC Future Fellow
Centre for Astrophysics & Supercomputing
Swinburne University of Technology
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20210311/8b47a14d/attachment.html>