[Difx-users] Question of the loss in Int. time and the phase shift obtained from the correlations

Sun May 21 22:39:05 EDT 2017

Hi Lupin,

OK, I think I partly understand now.  But so you've got two datastreams
which you've generated from noise sources, is that right?  They have
independent noise and a common signal which has a small but non-zero
delay?  And then you're putting both data streams into DiFX and saying they
are from the same physical position, so they get the same delay model
applied.  How are you calculating the single phase number?  Is that a
simple average over the whole band?  The changing amplitudes due to the
requantisation plus the phase ripple could easily account for some of the
differences.  It also seems like your S/N is absurdly high, which is
probably exacerbating a lot of the quantisation-related effects.  What if
you reduce the amplitude of the common noise source to something a bit more
realistic, like a maximum of 1% of the independent noise amplitude?

I don't have an explanation for why the phase differs between an LO of
86000 MHz vs 220000 MHz when you're extracting a 128 MHz subband, but I can
only point the finger at quantisation + numerical precision effects coupled
with the very high degree of correlation that appears to be present in your
setup.

Cheers,
Adam

On 22 May 2017 at 12:12, Lupin Lin <lupin at asiaa.sinica.edu.tw> wrote:

> Dear Adam,
>
> Thanks for your repsonse to give the indication.
> For the 1st question, I will further check with the expert of HOPS
> developer.
>
> For my 2nd question, the Jan’s code is “m5subband” provided in the latest
> DiFX version to extract the specific frequency range information from a
> wide/full band data.
> The original input is also the 2-bit VDIF file.
> I just compare with "the cross-correlation with two 2-bit wide band VDIF
> files at a specific frequency range using the zoom-band mode" and "one data
> subtracted a the same frequency range to cross-correlate another wide band
> file supported with zoom-band mode.”
> A simple expression in the .v2d file is the comparison between:
> (XXXXX and OOOOO are data with 2 GHz, and SSSSS is a data with the band of
> 256 MHz subtracted from XXXXX)
> ————————————————————————
> ANTENNA G1
> {
>    file = XXXXX.vdif
>    format = VDIF/8224/2
>    phaseCalInt = 0
>    addZoomFreq= freq at 230000.00/bw at 256.0
> }
> ANTENNA G2
> {
>    file = OOOOO.vdif
>    format = VDIF/8224/2
>    phaseCalInt = 0
>    addZoomFreq= freq at 230000.00/bw at 256.0
> }
> ————————————————————————  And,
> ANTENNA G1
> {
>    file = SSSSS.vdif
>    format = VDIF/8224/2
>    phaseCalInt = 0
> }
> ANTENNA G2
> {
>    file = OOOOO.vdif
>    format = VDIF/8224/2
>    phaseCalInt = 0
>    addZoomFreq= freq at 230000.00/bw at 256.0
> }
> ————————————————————————
>
> The loss in SNR or the amplitude is expected due to the re-sampling (FFT
> and IFFT for the original XXXXX.vdif) of the data, but we do not clearly
> understand the origin to cause the "possible shift" in the obtained phase
> distribution.
> (We suspect the shift in phase is due to the adoption of the Ref. Freq.,
> but the series of my tests cannot confirm this point.)
>
> The motivation to do it is that we have limited/very slow internet speed
> in GLT.
> In order to check the preliminary result via the obtained observation, we
> would like to send only the band reduced data with a much smaller file size.
> So to investigate/handle the comparison between the aforementioned two
> cases is required for GLT team.
>
> Sincerely,
> --
> Lupin Chun-Che Lin
> Supporting Scientist of GLT (GreenLand Telescope) project
> in Institute of Astronomy and Astrophysics, Academia Sinica,
> 14F of Astronomy-Mathematics Building (Rm: 1405),
> National Taiwan University.
> No.1, Sec. 4, Roosevelt Rd, Taipei 10617, Taiwan.
> Tel: +886-2-2366-5464 <+886%202%202366%205464>
> Fax: +886-2-2367-7849 <+886%202%202367%207849>
>
>
>
> Adam Deller <adeller at astro.swin.edu.au> 於 2017年5月22日 上午8:15 寫道：
>
> Hi Lupin,
>
> On 19 May 2017 at 13:22, Lupin Lin <lupin at asiaa.sinica.edu.tw> wrote:
>
>> To the experienced user of DiFX or HOPS,
>>
>> Sorry to spend you some time on two questions.
>> The first one is related to the integration time presented on the final
>> plot after the auto- or cross- correlations.
>> (The DiFX I used is trunk ver. and and the HOPS ver. I used is 3.13)
>>
>> 1.
>> Here the GLT team recorded two signals with 3 noise sources (only 1 is
>> the same with a potential delay less than 1 ns) from the same ROACH, and
>> made a test in correlation.
>> We simply assumed that the Sky (Ref.) frequency is 230 GHz and the data
>> were obtained from the same site (both G1 and G2 were assigned to the same
>> position of GLT).
>> We correlated the data of 5-sec. supported with the zoom-band mode at
>> 1,200-1,328 MHz.
>> ————(Contents in .v2d file)———
>> vex = ext.vex.obs
>> mjdStart = 2017y124d05h42m00s
>> mjdStop = 2017y124d05h42m05s
>> antennas = G1,G2
>> startSeries = 7000
>> dataBufferFactor = 16
>> visBufferLength = 80
>> nDataSegments = 8
>> minLength = 0.5
>> ————————————————
>> But according to the attached figure of  “Zoom128_5sec.pdf”, the int time
>> is only 4.995 sec.
>> What’s the reason to cause a loss of the obtained Int. time?
>>
>>
> This may be HOPS-related: I'm not an expert on HOPS so I can't comment.
> But another possibility is that the correlation runs from 054200 to 054205
> at the geocentre, and if the recorded data duration was only 5 seconds long
> but at the telescope, then the geometric delay will mean than a few
> milliseconds of data will be chopped off.  Maybe one of the HOPS users can
> chime in on how that integration time number is calculated in the HOPS plot
> - is it based on the weight?  If so then my hypothesis is plausible.
>
>
>>
>> 2.
>> Following the previous test, I used Jan Wagner’s code to do the band
>> subtraction (the original full band is 2GHz) and checked the obtained phase
>> of different cases.
>> I summarized my tests and my questions in the attached file “phase.pdf”.
>>
>>
>> The 1st case is the same as shown in the attached “Zoom128_5sec.pdf”.
>> What’s the possible origin to cause the phase shift?
>> I originally suspect that such a shift is originated from the change in
>> Sky/Ref. freq, but it might be wrong….
>>
>>
> I'm sorry, but I can't follow what your are trying to do here. I'm not
> sure what Jan's code is that you're referring to (is it something that
> filters e.g. your 2 GHz band down to 128 MHz in a chosen window?) If that
> is the case, what is the input - already quantised 2 bit VDIF data?  That
> will lead to nasty re-quantisation errors - you'll notice that the
> amplitude goes down, too.
>
> But the root of the problem is: what are you actually ultimately trying to
> achieve?  A verification of some kind?
>
> Cheers,
> Adam
>
>
>> Sincerely,
>> --
>> Lupin Chun-Che Lin
>> Supporting Scientist of GLT (GreenLand Telescope) project
>> in Institute of Astronomy and Astrophysics, Academia Sinica,
>> 14F of Astronomy-Mathematics Building (Rm: 1405),
>> National Taiwan University.
>> No.1, Sec. 4, Roosevelt Rd, Taipei 10617, Taiwan.
>> Tel: +886-2-2366-5464 <+886%202%202366%205464>
>> Fax: +886-2-2367-7849 <+886%202%202367%207849>
>>
>> _______________________________________________
>> Difx-users mailing list
>> Difx-users at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>>
>>
>
>
> --
> !=============================================================!
> Dr. Adam Deller
> ARC Future Fellow, Senior Lecturer
> Centre for Astrophysics & Supercomputing
> Swinburne University of Technology
> John St, Hawthorn VIC 3122 Australia
> phone: +61 3 9214 5307 <+61%203%209214%205307>
> fax: +61 3 9214 8797 <+61%203%209214%208797>
>
> office days (usually): Mon-Thu
> !=============================================================!
>
>
>

-- 
!=============================================================!
Dr. Adam Deller
ARC Future Fellow, Senior Lecturer
Centre for Astrophysics & Supercomputing
Swinburne University of Technology
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20170522/631d9bc9/attachment-0001.html>