[Difx-users] vex2difx bandwidth-sample rate check

Mon Jun 14 20:45:20 EDT 2021

Hi Joe, Adam

I cannot remember if the handing of the sample rate as described is just something I overlooked or a more fundamental issue. It may well be just something I overlooked.

Internally mpifxcorr certainly uses the “correct” sampling rate for complex data. It would have been easier to just lie about the sampling rate entirely (less code to tweak)  but I felt it would be something that would cause subtle bugs in the future if not handled correctly.

For the vex file, I just use the sampling rate of double the bandwidth. Vex2difx definitely DOES handle data labelled as complex on the vex file, but this is probably just using my own non-standard and un-documented extension to vex 1.5.

If you have a .vex file with the station schedules as real sampled Mark5B (thats a typical. Initial setup), then there is a perl script “addVDIF.pl” which comes with DIFX which will changes the setup to VDIF, and has a “complex” option. Note addVDIF.pl depends on Astro::Vex, which comes with DIFX but can be a pain to install because of possible conflicts between vex and perl Lex parser.  The —perl option option on the install-diff script should install all of this.

addVDIF has the following options:

 -bits <N>       Set bit depth to N (default 2)
 -framesize <N>  Set frame size to N (default 8032)
 -complex        Complex samples

So you can set appropriate #bits and label as complex.

The main this this does is set  the track format in $TRACK section:

  track_frame_format = VDIFC/8032/2;

VDIFC is complex VDIF data in “Single Sideband” mode. If you have “Double Sideband” mode, you need VDIFD (from memory). addVDIF.pl does not support that, but it would be a trivial change.

So essentially, if you just leave sample rate a 2xbandwidth and use the above track_frame_format in the vex file, it should “just work”.

Note you need trunk DIFX for all combinations of complex, USB/LSB, single/double sideband to work. Lots are broken in current release.

We probably should re-assess the notes on the wiki and update. For example notes on VDIF looks out of date and are buried near the bottom of the Documentation page after a lot of much less important stuff.

Cheers
Chris

On Jun 15, 2021, at 8:40 AM, Adam Deller via Difx-users <difx-users at listmgr.nrao.edu<mailto:difx-users at listmgr.nrao.edu>> wrote:

Hi Joe,

Right, I see.  So if you leave the code and vex file untouched and just change the ANTENNA entries in the v2d file (add sampling=COMPLEX inside the ANTENNA blocks), I think you should be ok.  What is the data format?  Does that match the example, or do you need to override that too?

Cheers,
Adam

On Tue, 15 Jun 2021 at 00:21, Joe Skeens <jskeens1 at utexas.edu<mailto:jskeens1 at utexas.edu>> wrote:
Hi Adam,

Thanks for the help and the quick response! We've been using an adapted version of the files given in the pre-packaged tests that come with DiFX to establish a proof-of-concept run with simulated complex data. That is to say, I'm changing the premade .vex file by hand to match the properties of the simulated signals I'm feeding into the correlator. Prior to removing the if statement I mentioned, I was giving a bandwidth of sample rate divide by 2 in the .vex file, running vex2difx, then manually changing the bandwidth to the sample rate in the .input file before running mpifxcorr. I'm not sure if that leads to a problem somewhere in the guts of DiFX, but we've been able to see injected tones in the output FITS file using this method.

Best,
Joe

On Sat, Jun 12, 2021 at 4:56 AM Adam Deller <adeller at astro.swin.edu.au<mailto:adeller at astro.swin.edu.au>> wrote:
Hi Joe,

I don't believe the current vex standard is actually capable of representing that a signal is complex sampled (a problem which is one of the justifications for the long-awaited vex2 standard).  To sidestep this, the v2d file can state that the data sampling is complex - but the vex loader doesn't have access to that information.  So the vex file basically can't represent the full info, and it has to be wrong or at least incomplete in one way or another.  So if you just change the sample rate in the vex file (but still tell vex2difx via the v2d file that the sampling is complex) you should be ok.

Out of interest, how did you get this vex file (with the sampling rate set to the bandwidth) in the first place?

Cheers,
Adam

On Sat, 12 Jun 2021 at 03:22, Joe Skeens via Difx-users <difx-users at listmgr.nrao.edu<mailto:difx-users at listmgr.nrao.edu>> wrote:
Hello all,

I've been working with a couple of collaborators on running DiFX with
complex signals, and we've found something that looks like it may be an
oversight. In the vex2difx file 'vexload.cpp,' there's a simple if
statement check which seems to ensure the sample rate is above the Nyquist frequency:

if(bandwidth - stream.sampRate/2 > 1e-6)

This causes vex2difx to exit on an error condition for bandwidth greater
than sample rate divided by 2, but for a complex signal, the signal is
only undersampled for bandwidth greater than sample rate. Commenting out
this check and rebuilding DiFX seems to correct the problem. Is there
something I'm missing with respect to how the bandwidth is processed in
the channel definitions under $FREQ in the .vex file, or has this check
been designed with only real signals in mind? Thanks in advance for any
insights.

Best,
Joe Skeens
_______________________________________________
Difx-users mailing list
Difx-users at listmgr.nrao.edu<mailto:Difx-users at listmgr.nrao.edu>
https://listmgr.nrao.edu/mailman/listinfo/difx-users

--
!=============================================================!
A/Prof. Adam Deller
ARC Future Fellow
Centre for Astrophysics & Supercomputing
Swinburne University of Technology
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!

--
!=============================================================!
A/Prof. Adam Deller
ARC Future Fellow
Centre for Astrophysics & Supercomputing
Swinburne University of Technology
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
_______________________________________________
Difx-users mailing list
Difx-users at listmgr.nrao.edu<mailto:Difx-users at listmgr.nrao.edu>
https://listmgr.nrao.edu/mailman/listinfo/difx-users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20210615/75ef36cd/attachment-0001.html>