[Difx-users] complex support

[Phillips, Chris (CASS, Marsfield)]

Hi Jayce

I'm working the data from the Long Wavelength Array and I am trying to see if we can move from our homegrown correlator to DiFX.  The two modes I need to support are a LWA-only mode with homogeneous data and an "eLWA" mode with heterogeneous data that combines the LWA stations with the VLA 4-band system.  For the LWA-only mode I have 4+4-bit complex voltage data from two 19.6 MHz wide spectral windows.  I am currently converting these data from the LWA format into two VDIF files, one per window, for correlation with DiFX.  My .input file describes the data as:
> 
> DATA FORMAT:        INTERLACEDVDIF/0:1
> QUANTISATION BITS:  4
> DATA FRAME SIZE:    7872
> DATA SAMPLING:      COMPLEX_DSB
> DATA SOURCE:        FILE
> 

I have never personally tested complex_dsb data with interlaced vdif. I have no idea if there could be any “interaction” with the VDIF interleaving and the specific data type (I would have thought not, but you never know).

The issue could also be with interpreting the interlaced VDIF, not the complex double sideband.

Once you have tested using trunk, not the released version, I would try and use some of the vdifio utilities to try and convert the interlaced VDIF into plain VDIF - either extracting out one thread, or merging into multichannel single thread VDIF. You would obviously need to update the DIFX .input (and or .v2d) file and vexfile to match the new setup.

I would also be using the m5access tools to do things like make the autocorrelation (both before and after fiddling with the data layout). Check DIFX gives the same auto shape as you get with m5spec.  If you use a modulated noise source, folding the data (m5fold) on the period of the noise cal can be very informative also.  Also look at the headers and check that the # frames/sec etc are what you expect (before and after merging threads etc). If you decide it is the interlacing is the issue, I suspect you will need to discuss with Walter.


> I am less sure about DiFX supporting our eLWA mode.  For this we have the same 4+4-bit complex voltage data from the LWA stations but with a bandwidth of 9.8 MHz.  We are combining this with 4-bit real voltages from the VLA with 8 MHz of bandwidth and a slightly different frequency setup.  We currently process these data by shifting the center frequency of the LWA data to match that of the VLA, running two F-engines with different channelization to arrive at the same channel width for both data sources, and then cross correlate what overlaps.  Is this mode something that DiFX supports?
> 

Can you represent this data as VDIF? If you you can probably handle the different setup using zoom bands. However if this can be done will really depend on the EXACT setup. Basically you need to be able to channelise the two setups with the same final frequency resolution (ie different FFT sizes will be needed, with the ratio of the FFT size matching the ratio of the bandwidth difference. But the sky frequency of the frequency points out of the FFT need to match exactly (ie even if you could achieve the same frequency resolution, if the fine channels are offset by 1/3 of a channel you will have problems (if that offset is a simple integer ratio, you can just use a higher frequency resolution)

I hope this makes sense.

Cheers
Chris