[Difx-users] complex support

[Jayce Dowell]

Hi Chris,

They should be.  I forgot to add that I also added a:

FREQ OFFSET 0 (Hz): 0.005588

to the VLA antenna entries.

Jayce
--
On 3/23/21 7:17 PM, Phillips, Chris (CASS, Marsfield) wrote:
> *  [EXTERNAL]*
> 
> **
> 
> Are those sky frequencies “correct”? I guess if all telescopes are the 
> same it probably is OK.
> 
> Cheers
> 
> Chris
> 
> *From: *Jayce Dowell <jdowell at unm.edu>
> *Date: *Wednesday, 24 March 2021 at 12:11
> *To: *Adam Deller <adeller at astro.swin.edu.au>
> *Cc: *Phillips, Chris (CASS, Marsfield) <Chris.Phillips at csiro.au>, 
> difxusers <difx-users at listmgr.nrao.edu>
> *Subject: *Re: [Difx-users] complex support
> 
> Hi Adam & Chris,
> 
> I've moved on to the VLA+LWA data and am running into a new problem.  I
> have my frequency setups as:
> 
> # FREQ TABLE #######!
> FREQ ENTRIES:       2
> FREQ (MHZ) 0:       71.09999999441
> BW (MHZ) 0:         9.80000000000
> SIDEBAND 0:         U
> NUM CHANNELS 0:     392
> CHANS TO AVG 0:     1
> OVERSAMPLE FAC. 0:  1
> DECIMATION FAC. 0:  1
> PHASE CALS 0 OUT:   0
> FREQ (MHZ) 1:       71.99999999441
> BW (MHZ) 1:         8.00000000000
> SIDEBAND 1:         U
> NUM CHANNELS 1:     320
> CHANS TO AVG 1:     1
> OVERSAMPLE FAC. 1:  1
> DECIMATION FAC. 1:  1
> PHASE CALS 1 OUT:   0
> 
> In my data streams table I have the LWA stations recording complex
> double sideband with frequency setup 0 and one zoom band of setup 1.
> The VLA antennas are set real upper sideband with frequency setup 1 and
> no zoom band.  I think this is what I want and mpifxcorr doesn't
> complain when I run it.  However, difx2fits immediately segfaults when I
> try to build a FITS-IDI file.  Any suggestions?
> 
> Jayce
> --
> On 3/10/21 4:19 PM, Adam Deller wrote:
>  > *  [EXTERNAL]*
>  >
>  > **
>  > Glad to hear it!
>  >
>  > On Thu, 11 Mar 2021 at 09:58, Jayce Dowell <jdowell at unm.edu
>  > <mailto:jdowell at unm.edu <mailto:jdowell at unm.edu>>> wrote:
>  >
>  >     Hi Chris,
>  >
>  >     After some digging I was able to get to the bottom of this and I 
> am now
>  >     getting results that I am happy with.  It was a combination of
>  >     switching
>  >     over to trunk and fixing a bad assumption in my VDIF conversion that
>  >     everyone represents four-bit complex integers in the same way.
>  >
>  >     Jayce
>  >     --
>  >     On 2/4/21 3:08 PM, Phillips, Chris (CASS, Marsfield) wrote:
>  >      >    [EXTERNAL]
>  >      >
>  >      > Hi Jayce
>  >      >
>  >      > I'm working the data from the Long Wavelength Array and I am
>  >     trying to see if we can move from our homegrown correlator to DiFX.
>  >     The two modes I need to support are a LWA-only mode with homogeneous
>  >     data and an "eLWA" mode with heterogeneous data that combines the
>  >     LWA stations with the VLA 4-band system.  For the LWA-only mode I
>  >     have 4+4-bit complex voltage data from two 19.6 MHz wide spectral
>  >     windows.  I am currently converting these data from the LWA format
>  >     into two VDIF files, one per window, for correlation with DiFX.  My
>  >     .input file describes the data as:
>  >      >>
>  >      >> DATA FORMAT:        INTERLACEDVDIF/0:1
>  >      >> QUANTISATION BITS:  4
>  >      >> DATA FRAME SIZE:    7872
>  >      >> DATA SAMPLING:      COMPLEX_DSB
>  >      >> DATA SOURCE:        FILE
>  >      >>
>  >      >
>  >      > I have never personally tested complex_dsb data with interlaced
>  >     vdif. I have no idea if there could be any “interaction” with the
>  >     VDIF interleaving and the specific data type (I would have thought
>  >     not, but you never know).
>  >      >
>  >      > The issue could also be with interpreting the interlaced VDIF,
>  >     not the complex double sideband.
>  >      >
>  >      > Once you have tested using trunk, not the released version, I
>  >     would try and use some of the vdifio utilities to try and convert
>  >     the interlaced VDIF into plain VDIF - either extracting out one
>  >     thread, or merging into multichannel single thread VDIF. You would
>  >     obviously need to update the DIFX .input (and or .v2d) file and
>  >     vexfile to match the new setup.
>  >      >
>  >      > I would also be using the m5access tools to do things like make
>  >     the autocorrelation (both before and after fiddling with the data
>  >     layout). Check DIFX gives the same auto shape as you get with
>  >     m5spec.  If you use a modulated noise source, folding the data
>  >     (m5fold) on the period of the noise cal can be very informative
>  >     also.  Also look at the headers and check that the # frames/sec etc
>  >     are what you expect (before and after merging threads etc). If you
>  >     decide it is the interlacing is the issue, I suspect you will need
>  >     to discuss with Walter.
>  >      >
>  >      >
>  >      >> I am less sure about DiFX supporting our eLWA mode.  For this we
>  >     have the same 4+4-bit complex voltage data from the LWA stations but
>  >     with a bandwidth of 9.8 MHz.  We are combining this with 4-bit real
>  >     voltages from the VLA with 8 MHz of bandwidth and a slightly
>  >     different frequency setup.  We currently process these data by
>  >     shifting the center frequency of the LWA data to match that of the
>  >     VLA, running two F-engines with different channelization to arrive
>  >     at the same channel width for both data sources, and then cross
>  >     correlate what overlaps.  Is this mode something that DiFX supports?
>  >      >>
>  >      >
>  >      > Can you represent this data as VDIF? If you you can probably
>  >     handle the different setup using zoom bands. However if this can be
>  >     done will really depend on the EXACT setup. Basically you need to be
>  >     able to channelise the two setups with the same final frequency
>  >     resolution (ie different FFT sizes will be needed, with the ratio of
>  >     the FFT size matching the ratio of the bandwidth difference. But the
>  >     sky frequency of the frequency points out of the FFT need to match
>  >     exactly (ie even if you could achieve the same frequency resolution,
>  >     if the fine channels are offset by 1/3 of a channel you will have
>  >     problems (if that offset is a simple integer ratio, you can just use
>  >     a higher frequency resolution)
>  >      >
>  >      > I hope this makes sense.
>  >      >
>  >      > Cheers
>  >      > Chris
>  >      >
>  >
>  >
>  >
>  > --
>  > !=============================================================!
>  > A/Prof. Adam Deller
>  > ARC Future Fellow
>  > Centre for Astrophysics & Supercomputing
>  > Swinburne University of Technology
>  > John St, Hawthorn VIC 3122 Australia
>  > phone: +61 3 9214 5307
>  > fax: +61 3 9214 8797
>  >
>  > office days (usually): Mon-Thu
>  > !=============================================================!
>