[Difx-users] Missing baselines with R-X/Y data

Wed Jun 2 20:28:24 EDT 2021

Hi Eskil,

Thanks for the update.  Glad to hear you got a resolution, and sorry I
hadn't had a chance to try to reproduce the problem and go digging with you!

Cheers,
Adam

On Wed, 2 Jun 2021 at 15:29, Eskil Varenius via Difx-users <
difx-users at listmgr.nrao.edu> wrote:

> Hi all,
>
> In case anyone is interested, here is a bried update on this issue
> (missing baselines in mixed-pol setup when not using zoombands): Since a
> while ago, I believe the problem is not the mix of polarisations, but
> rather the mix of USB/LSB. The ON station is recording USB BBCs while the
> OE and OW stations record LSB BBCs. I have been told there is a bug/feature
> in difx2mark4 with mixed USB/LSB data, where difx supposedly writes two
> frequency tables, but difx2mark4 only reads the baselines matching one of
> them. This is a likely explanation for why I seem to lose the OE-OW
> baseline; it is the only LSB-LSB configuration, while the USB-LSB baselines
> (OE-ON and OW-ON) are kept. Using zoombands, it seems the output is forced
> into one frequency table and thus no problem for difx2mark4 et al.
>
> Kind regards
>
> Eskil
> On 2021-02-04 18:09, Eskil Varenius wrote:
>
> Hi Adam, all
>
>
>
> Finally some data so I can return to the mixed-pol issue again. To remind
> you, my problem is that when correlating R-pol (1Gbps) data from ONSALA60
> against X/Y-pol (2Gbps) data from ONSA13NE and ONSA13SW, I must define
> “zoom bands” to get visibilities on all baselines. Not sure if the problem
> is in DiFX, or difx2mark4 (or elsewhere).  I have uploaded a tar.gz archive
> with input/output files (excluding 18GB of VDIF data) at
>
> https://chalmersuniversity.box.com/s/9ed0t61bk04ag88is2ovhlm0er4qd00n
>
> There you can check logfiles, input files etc. If you want the data, I can
> share it (no time right now to remember how to split out a small chunk of
> data for testing, so used one full 30 sec scan). I can also try various
> things on my end if you have suggestions.
>
>
>
> Here are the steps to reproduce with DiFX 2.6.2:
>
>
>
> Unzip archive,
>
> Edit "*.files" with correct paths to "data" directory
>
> cd zoom
>
> Edit "on1028_1.machines" accordingly (e.g. computer name)
>
> vex2difx -v -v -v -d *v2d --force (need force as this is mixed
> circular/linear data)
>
> calcif2 *.calc
>
> startdifx -n -f *.input -v
>
> I now get messages, see logfile, like
>
> "Thu Feb  4 17:56:58 2021   0 skirner WARNING  Baseline 0 frequency 0
> points at two different frequencies that are apparently identical - this is
> not wrong, but very strange.  Check the input file"
>
> The message is correct, I have identical zoom windows - otherwise I don't
> get all baselines. To see result, I run
>
> difx2mark4 -v -b X 2300 14000 *.difx -s ../station_code_file.txt
>
> and check the output on screen. All baselines are included:
>
> [...]
>
>       n120[SX] 448
>
>       n120[ST] 896
>
>       n120[XT] 448
>
>       n120[SS] 448
>
>       n120[XX] 448
>
>       n120[TT] 448
>
> [...]
>
>
>
> Same procedure without zoom-windows, I get a different set of baselines
>
>
>
> [...]
>
>       n120[SX] 448
>
>       n120[XT] 448
>
>       n120[SS] 448
>
>       n120[XX] 224
>
>       n120[TT] 448
>
> [...]
>
> Curiously, the "ST" (which is the ONSA13NE-SW baseline) is missing, and
> the XX (ONSALA60) is half (224).
>
>
>
> Not sure the zoom-window version is correct, but I need the ST-baseline,
> and it seems to work, so I use zoom-windows. But, I'm curious: why does
> this happen?
>
>
>
> Kind regards
>
> Eskil
>
>
>
> *From: *Difx-users <difx-users-bounces at listmgr.nrao.edu>
> <difx-users-bounces at listmgr.nrao.edu> on behalf of Eskil Varenius via
> Difx-users <difx-users at listmgr.nrao.edu> <difx-users at listmgr.nrao.edu>
> *Reply to: *Eskil Varenius <eskil.varenius at chalmers.se>
> <eskil.varenius at chalmers.se>
> *Date: *Monday, 30 November 2020 at 11:51
> *To: *Adam Deller <adeller at astro.swin.edu.au> <adeller at astro.swin.edu.au>
> *Cc: *"difx-users at listmgr.nrao.edu" <difx-users at listmgr.nrao.edu>
> <difx-users at listmgr.nrao.edu> <difx-users at listmgr.nrao.edu>
> *Subject: *Re: [Difx-users] A few surprises
>
>
>
> Hi Adam,
>
> Apologies, but I will have to defer sending additional details as the data
> in question have already been removed after correlation. (I would like to
> verify that the files I send indeed produce the errors I claim, so I don't
> send files with other typos). Also, I have too many other todo-items to
> follow this up right now, but we regularly observe with similar settings so
> I should get another data set to play with in a few weeks. When I do, I
> will check the input-file differences and let you know, and keep the data
> on disk for additional tests. But I did experience the same thing for all
> similar data so far. Will let you know once I have new data.
>
> Cheers
>
> Eskil
>
> On 2020-11-30 11:43, Adam Deller wrote:
>
> Hi Eskil,
>
>
>
> Thanks for the heads up, the first two are surprising (I'm never surprised
> by gaps in documentation!)
>
>
>
> I've got no experience with the intel compilers, so will pass on that
> one.  For the second, what is the difference in the .input file produced by
> vex2difx in the two cases?  That might help shed some light.
>
>
>
> Cheers,
>
> Adam
>
>
>
> On Mon, 30 Nov 2020 at 21:18, Eskil Varenius via Difx-users <
> difx-users at listmgr.nrao.edu> wrote:
>
> Dear difx developers,
> I have noted a few surprises which I would like to report:
>
> - Compiling DiFX 2.6.1/2.6.2 with intel 2019 compiler, mpifxcorr runs
> but refuses to actually read any data. It just sits there, never
> starting the correlation. Using intel 2018 compiler instead, everything
> works nicely.
>
> - Correlating mixed-polarisation data (antenna ON: R-pol, OE and OW: X/Y
> pol) delivers correlation products on the mixed circ/linear baselines
> (R-X and R-Y pol), but not on the linear/linear baselines (XX,XY,YX,YY).
> However, if I add zoomwindows - describing the same setup already in the
> VEX file, so no actual zooming - then I get also the linear polarisation
> products. (I know mixed-pol support is limited, but might as well note
> the behaviour.)
>
> - "difx2mark4 --help" v1.6 does not say it needs .calc files, but seems
> it does. So documentation should be amended?
>
> Kind regards
> Eskil
>
> _______________________________________________
> Difx-users mailing list
> Difx-users at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>
>
>
>
> --
>
> !=============================================================!
>
> A/Prof. Adam Deller
>
> ARC Future Fellow
>
> Centre for Astrophysics & Supercomputing
>
> Swinburne University of Technology
> John St, Hawthorn VIC 3122 Australia
>
> phone: +61 3 9214 5307
>
> fax: +61 3 9214 8797
>
>
>
> office days (usually): Mon-Thu
> !=============================================================!
>
> _______________________________________________
> Difx-users mailing list
> Difx-users at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>

-- 
!=============================================================!
A/Prof. Adam Deller
ARC Future Fellow
Centre for Astrophysics & Supercomputing
Swinburne University of Technology
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20210603/6a569314/attachment-0001.html>