<div dir="ltr">Hi Richard,<div><br></div><div>Looks like there is a problem mid-file, and when it tries to re-sync the header it finds is corrupted. I can suggest a couple of things to try:</div><div><br></div><div>you can run countVDIFpackets (a utility in vdifio) which is probably slower than vdifsummary (what utility is this? I'm not aware of a "vdifsummary", there is a "vsum"...?) and is pretty basic but actually does check for every packet, and prints a message every time a problem is seen. That might give you some extra clues, so I'd try that first. And if you really want to get blasted away by lots of logging, you can use printVDIF, which prints a little summary of each and every packet header. You could pipe that to grep to look for anomalies. </div><div><br></div><div>Looks like the problem is very early in the file, so if you dd the first second or so and put it on an ftp server somewhere, I could also take a look.</div><div><br></div><div>Cheers,</div><div>Adam</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 22, 2016 at 10:57 AM, Richard Dodson <span dir="ltr"><<a href="mailto:richard.dodson@uwa.edu.au" target="_blank">richard.dodson@uwa.edu.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_extra">Dear All
</div><div class="gmail_extra"><br></div><div class="gmail_extra"> I have one of the usual nightmare twisted DiFX correlation problems.</div><div class="gmail_extra"> </div><div class="gmail_extra"> I am trying to use DiFX on VDIF data which has been copied off the VERA OCTAVE systems (and similar) and converted. </div><div class="gmail_extra"><br></div><div class="gmail_extra"> The problem is almost certainly in the data copying -- but I need to provide some feedback on what is wrong for it to be fixed</div><div class="gmail_extra"><br></div><div class="gmail_extra"> The first problem that I found was in the VDIF file: all the invalid flags were set, the number of channels was wrong and the date was wrong by 1 day. :(</div><div class="gmail_extra"><br></div><div class="gmail_extra"> Jan has a program to fix all of these :) -- but he is not around to check if I have used this correctly :( :(</div><div class="gmail_extra"><br></div><div class="gmail_extra"> After these fixes the correlation runs, but the data file is empty. What messages should I be checking to work out what is happening? I append some messages which look suspicious but don't convey any information to me ... </div><div class="gmail_extra"><br></div><div class="gmail_extra"> All the best</div><div class="gmail_extra"> Richard</div><div class="gmail_extra"><br></div><div class="gmail_extra">Comments:</div><div class="gmail_extra"> vdifsummary reports seem OK</div><div class="gmail_extra"><br># vdifsummary /lustre/kjcc/k16mk02f/MIZ/k16m<wbr>k02f_kava_miz.vdif <br>[1:1] check k16mk02f_kava_miz.vdif -> Good! it is a VDIF data scan -> add to 1<br>k16mk02f_kava_miz.vdif 4,108,790,400,000 31317 sec( 8:41:57) 57467 Mar 20 2016y080d 11:00:03 - 19:41:59 1312 100000<br>3,827 GB(= 3.7 TB)(= 4,108,790,400,000 B)<br><br>Log messages which might be relevant:<br></div><div class="gmail_extra"><div><br></div><p><span></span></p><p><span></span></p>
<p><span>2016-08-22 16:30:32,548 DiFXAlert INFO MPI[ 1] compute-0-28.local k16mk02f_1 Datastream 1 has opened file index 0, which was /lustre/kjcc/k16mk02f/MIZ/k16m<wbr>k02f_kava_miz.vdif</span></p>
<p><span>2016-08-22 16:30:32,548 DiFXAlert VERBOSE MPI[ 2] compute-0-28.local k16mk02f_1 input.bad() is 0, input.fail() is 0</span></p>
<p><span>2016-08-22 16:30:32,700 DiFXAlert ERROR MPI[ 1] compute-0-28.local k16mk02f_1 Lost Sync on segment 1! Will attempt to resync. Deltatime was -1.13239e+09</span></p>
<p><span>2016-08-22 16:30:32,701 DiFXAlert INFO MPI[ 1] compute-0-28.local k16mk02f_1 Config has changed!</span></p>
<p><span>2016-08-22 16:30:32,702 DiFXAlert INFO MPI[ 1] compute-0-28.local k16mk02f_1 After regaining sync, the frame start day is 70573, the frame start seconds is 70631, the frame start ns is -2147483648, readscan is 2, readseconds is 1132388471, readnanoseconds is -2147483648</span></p></div><div class="gmail_extra"> note the 2^31 values !!!!</div></div>
<br>______________________________<wbr>_________________<br>
Difx-users mailing list<br>
<a href="mailto:Difx-users@listmgr.nrao.edu">Difx-users@listmgr.nrao.edu</a><br>
<a href="https://listmgr.nrao.edu/mailman/listinfo/difx-users" rel="noreferrer" target="_blank">https://listmgr.nrao.edu/<wbr>mailman/listinfo/difx-users</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr">!=============================================================!<br>Dr. Adam Deller </div><div dir="ltr">Ph +31 521595785 / Fax <span style="font-size:12.8000001907349px">+31 521595101</span><br>Staff Astronomer, Astronomy Group <br>ASTRON, Oude Hoogeveensedijk 4<br>7991 PD Dwingeloo,
The Netherlands<br>!=============================================================!</div></div></div></div>
</div>