[Difx-users] complex support

Adam Deller adeller at astro.swin.edu.au
Wed Mar 24 07:57:43 EDT 2021 

I'm guessing that the problem is, as Cormac suggested, that there are
LWA-LWA baselines that have a 9.8 MHz bandwidth, and difx2fits can't build
a valid FITS file with both those and 8 MHz bands.  printDiFXInput is a
good summary tool, which is easier than diving into the baseline table
directly and working backwards to figure out if you have any offending
baselines being correlated.

Cheers,
Adam

On Wed, 24 Mar 2021 at 22:43, Jan Wagner via Difx-users <
difx-users at listmgr.nrao.edu> wrote:

> Hi Jayce,
>
> you could use "printDiFXInput.py <inputfile>" from either DiFX 2.6.2
> or for more details DiFX Trunk. That will show you the list of
> datastreams and their recorded plus zoom frequencies, and a summary
> of the baseline table showing the frequencies that are expected to
> be found in the *.difx/DIFX_... output files.
>
> Similarly, "printDiFX.py <difxfile>" lists the visibility records in
> the mpifxcorr output files. Maybe some frequencies there are
> "unexpected"?
>
> Depending on how difx2fits was compiled and whether you have the gdb
> debugger installed, you could "gdb --args difx2fits -d test.fits"
> and type "run" in the gdb console. After the crash, type "bt" in the
> same gdb console. That might produce a more detailed explanation
> (for developers) of what could have provoked the segfault.
>
> regards,
> Jan
>
> Am 24.03.2021 um 02:10 schrieb Jayce Dowell via Difx-users:
> > Hi Adam & Chris,
> >
> > I've moved on to the VLA+LWA data and am running into a new
> > problem.  I have my frequency setups as:
> >
> > # FREQ TABLE #######!
> > FREQ ENTRIES:       2
> > FREQ (MHZ) 0:       71.09999999441
> > BW (MHZ) 0:         9.80000000000
> > SIDEBAND 0:         U
> > NUM CHANNELS 0:     392
> > CHANS TO AVG 0:     1
> > OVERSAMPLE FAC. 0:  1
> > DECIMATION FAC. 0:  1
> > PHASE CALS 0 OUT:   0
> > FREQ (MHZ) 1:       71.99999999441
> > BW (MHZ) 1:         8.00000000000
> > SIDEBAND 1:         U
> > NUM CHANNELS 1:     320
> > CHANS TO AVG 1:     1
> > OVERSAMPLE FAC. 1:  1
> > DECIMATION FAC. 1:  1
> > PHASE CALS 1 OUT:   0
> >
> > In my data streams table I have the LWA stations recording complex
> > double sideband with frequency setup 0 and one zoom band of setup
> > 1. The VLA antennas are set real upper sideband with frequency
> > setup 1 and no zoom band.  I think this is what I want and
> > mpifxcorr doesn't complain when I run it.  However, difx2fits
> > immediately segfaults when I try to build a FITS-IDI file.  Any
> > suggestions?
> >
> > Jayce
> > --
> > On 3/10/21 4:19 PM, Adam Deller wrote:
> >> *  [EXTERNAL]*
> >>
> >> **
> >> Glad to hear it!
> >>
> >> On Thu, 11 Mar 2021 at 09:58, Jayce Dowell <jdowell at unm.edu
> >> <mailto:jdowell at unm.edu>> wrote:
> >>
> >>     Hi Chris,
> >>
> >>     After some digging I was able to get to the bottom of this
> >> and I am now
> >>     getting results that I am happy with.  It was a combination of
> >>     switching
> >>     over to trunk and fixing a bad assumption in my VDIF
> >> conversion that
> >>     everyone represents four-bit complex integers in the same way.
> >>
> >>     Jayce
> >>     --
> >>     On 2/4/21 3:08 PM, Phillips, Chris (CASS, Marsfield) wrote:
> >>      >    [EXTERNAL]
> >>      >
> >>      > Hi Jayce
> >>      >
> >>      > I'm working the data from the Long Wavelength Array and I am
> >>     trying to see if we can move from our homegrown correlator to
> >> DiFX.     The two modes I need to support are a LWA-only mode
> >> with homogeneous
> >>     data and an "eLWA" mode with heterogeneous data that combines
> >> the
> >>     LWA stations with the VLA 4-band system.  For the LWA-only
> >> mode I
> >>     have 4+4-bit complex voltage data from two 19.6 MHz wide
> >> spectral
> >>     windows.  I am currently converting these data from the LWA
> >> format
> >>     into two VDIF files, one per window, for correlation with
> >> DiFX.  My
> >>     .input file describes the data as:
> >>      >>
> >>      >> DATA FORMAT:        INTERLACEDVDIF/0:1
> >>      >> QUANTISATION BITS:  4
> >>      >> DATA FRAME SIZE:    7872
> >>      >> DATA SAMPLING:      COMPLEX_DSB
> >>      >> DATA SOURCE:        FILE
> >>      >>
> >>      >
> >>      > I have never personally tested complex_dsb data with
> >> interlaced
> >>     vdif. I have no idea if there could be any “interaction” with
> >> the
> >>     VDIF interleaving and the specific data type (I would have
> >> thought
> >>     not, but you never know).
> >>      >
> >>      > The issue could also be with interpreting the interlaced
> >> VDIF,
> >>     not the complex double sideband.
> >>      >
> >>      > Once you have tested using trunk, not the released version, I
> >>     would try and use some of the vdifio utilities to try and
> >> convert
> >>     the interlaced VDIF into plain VDIF - either extracting out one
> >>     thread, or merging into multichannel single thread VDIF. You
> >> would
> >>     obviously need to update the DIFX .input (and or .v2d) file and
> >>     vexfile to match the new setup.
> >>      >
> >>      > I would also be using the m5access tools to do things like
> >> make
> >>     the autocorrelation (both before and after fiddling with the
> >> data
> >>     layout). Check DIFX gives the same auto shape as you get with
> >>     m5spec.  If you use a modulated noise source, folding the data
> >>     (m5fold) on the period of the noise cal can be very informative
> >>     also.  Also look at the headers and check that the #
> >> frames/sec etc
> >>     are what you expect (before and after merging threads etc).
> >> If you
> >>     decide it is the interlacing is the issue, I suspect you will
> >> need
> >>     to discuss with Walter.
> >>      >
> >>      >
> >>      >> I am less sure about DiFX supporting our eLWA mode.  For
> >> this we
> >>     have the same 4+4-bit complex voltage data from the LWA
> >> stations but
> >>     with a bandwidth of 9.8 MHz.  We are combining this with
> >> 4-bit real
> >>     voltages from the VLA with 8 MHz of bandwidth and a slightly
> >>     different frequency setup.  We currently process these data by
> >>     shifting the center frequency of the LWA data to match that
> >> of the
> >>     VLA, running two F-engines with different channelization to
> >> arrive
> >>     at the same channel width for both data sources, and then cross
> >>     correlate what overlaps.  Is this mode something that DiFX
> >> supports?
> >>      >>
> >>      >
> >>      > Can you represent this data as VDIF? If you you can probably
> >>     handle the different setup using zoom bands. However if this
> >> can be
> >>     done will really depend on the EXACT setup. Basically you
> >> need to be
> >>     able to channelise the two setups with the same final frequency
> >>     resolution (ie different FFT sizes will be needed, with the
> >> ratio of
> >>     the FFT size matching the ratio of the bandwidth difference.
> >> But the
> >>     sky frequency of the frequency points out of the FFT need to
> >> match
> >>     exactly (ie even if you could achieve the same frequency
> >> resolution,
> >>     if the fine channels are offset by 1/3 of a channel you will
> >> have
> >>     problems (if that offset is a simple integer ratio, you can
> >> just use
> >>     a higher frequency resolution)
> >>      >
> >>      > I hope this makes sense.
> >>      >
> >>      > Cheers
> >>      > Chris
> >>      >
> >>
> >>
> >>
> >> --
> >> !=============================================================!
> >> A/Prof. Adam Deller
> >> ARC Future Fellow
> >> Centre for Astrophysics & Supercomputing
> >> Swinburne University of Technology
> >> John St, Hawthorn VIC 3122 Australia
> >> phone: +61 3 9214 5307
> >> fax: +61 3 9214 8797
> >>
> >> office days (usually): Mon-Thu
> >> !=============================================================!
> >
> > _______________________________________________
> > Difx-users mailing list
> > Difx-users at listmgr.nrao.edu
> > https://listmgr.nrao.edu/mailman/listinfo/difx-users
>
>
> _______________________________________________
> Difx-users mailing list
> Difx-users at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>


-- 
!=============================================================!
A/Prof. Adam Deller
ARC Future Fellow
Centre for Astrophysics & Supercomputing
Swinburne University of Technology
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20210324/b1748c78/attachment-0001.html>

More information about the Difx-users mailing list