[Difx-users] difx2mark4 bug {External}

Tue Feb 13 13:08:37 EST 2024

Dear colleagues,

   Can you clarify, is there a convention on a case of two-letter station 
codes:
gs, GS, or Gs?

Leonid

> On 2024-02-13 11:56, John Barrett via Difx-users wrote:
> Hi Alexey,
> 
> I'm not entirely sure why the lowercase convention was enforced
> originally.  It is possibly due to the fact that DiFX takes all
> 2-letter codes and capitalizes them (e.g. Wf becomes WF) in the input
> file?
> So while your patch works for this particular case (JL (vex)-> JL
> (difx) -> JL (mk4)) it changes the convention used by most other sites
> where it is assumed (Wf (vex) -> WF (difx) -> Wf (mk4) ), and will
> break
> the ovex/root file that is generate (e.g. $FREQ section will no longer
> map to the proper station codes). The vex 'parser' (and I used that
> word very lightly) in difx2mark4 is extremely primitive and
> effectively is just looking for entire sections of the vex file to cut
> and paste into the ovex/root file, so it is not sophisticated enough
> to figure out all of the instances of a station 2-character code and
> replace them with what the user may actually desire.
> 
> I have forked the difx-dev branch here:
> 
> https://github.com/jpbarrett/difx.git
> 
> and added a patch which will introduce the behavior you want for just
> the stations specified on the command line (the default behavior will
> remain as it is now), using a new option "-c" or "--case-keep".
> 
> So for example if you run:
> 
> difx2mark4 -c GS -c WF -b X 0 10000000 -s ./station.codes -e 8899 -o
> -v ./h_1000.difx
> 
> And pass the station codes you want to preserve as all upper case,
> with the "-c" option, you will see a message to the effect of:
> 
>    processing $GLOBAL block
>       processing $EXPER block
>       processing $MODE block
>       Did not lower case-ify the station code: GS
>       Did not lower case-ify the station code: WF
>       processing $STATION block
>       processing $ANTENNA block
> 
> And the output ovex/root file "should" preserve the station code cases
> in the output.
> 
> I actually don't have any data which has station codes which are all
> upper case, so I can't tell if this works properly for your situation
> or not. If you get a chance, can you clone and compile this version of
> difx2mak4 and let me know if it works for the "JL" station situation?
> If it does, I will put in a pull request for this change to merge it
> to the dev branch.
> 
> I will also note, that if the -c code is erroneously applied to a
> station which in fact has a upper-lower case code in the vex file,
> then you will end up with a broken ovex/root file and fourfit will
> refuse to run on it as follows:
> 
> fourfit -c ./cf_3686_GEHSVY_pstokes2  -b GE -P XX ./8899/105-1800/
> fourfit: missing FREQ section for station G
> fourfit: Error filling in station parameters
> fourfit: Failure parsing observe vex portion of
> '/home/barrettj/work/projects/hops-git/build/test_data/vt9105/8899/105-1800//0016+731.341E5I'
> fourfit: The above errors occurred while processing
> fourfit: ./8899/105-1800//0016+731.341E5I
> fourfit: the top-level resolution is as follows:Error reading root for
> file ./8899/105-1800/, skipping
> 
> Let me know how it goes,
> 
> Best,
> 
> John
> 
> On 2/12/24 5:56 PM, Geoffrey B. Crew wrote:
> 
>> John (in CC) was looking at the issue, and as you've discovered,
>> it's not quite as trivial as you thought.
>> 
>> My recollection is that it is in fact a "standard" that two-letter
>> codes have this camel case (first letter upper and second lower)
>> convention, but I don't know how strict that is. Clearly the
>> difx2mark4 code is trying to be correct here.
>> 
>> A confusing issue (to me anyway) is that the vex2difx process seems
>> to elevate both to uppercase which I suspect is for historical
>> reasons (rather than an optimal design choice).
>> 
>> Anyway, if you can modify your vex to use 'Jl' rather than 'JL' you
>> should have no issues.
>> 
>> On 2/12/24 17:29, Alexey Melnikov via Difx-users wrote:
>> Dear all,
>> 
>> I need to withdraw my patch (it broke other cases) and ask for
>> advise. In
>> DiFX *.input files all 2-symbol station names are converted to
>> capitals
>> dispite how they were written in *.vex.
>> 
>> In my particular case, difx2mark4 fails since I have station named
>> in capitals
>> in vex, like 'JL'. So after line of code patched in my patch in
>> turns out to
>> 'Jl' and ovex file become wrong.
>> 
>> Does it mean that the better solution would be to keep 2-symbol
>> station names
>> in capitals when executing difx2mark4? Of course, another solution
>> is to keep
>> 2-symbol station names in sched's station.dat file as 'Jl', not
>> 'JL'.
>> 
>> regards,
>> Alexey
>> 
>> On 20240212 17:57:56, Adam Deller via Difx-users wrote:
>> Thanks, Alexey.
>> 
>> Hopefully someone from Haystack (Geoff, John?) could take a look at
>> this
>> and merge if happy? We should aim to get a new release off the git
>> repo in
>> the not too distant future, as the number of small bugs fixed is
>> starting
>> to stack up.
>> 
>> Cheers,
>> Adam
>> 
>> On Mon, 12 Feb 2024 at 11:57, Alexey Melnikov via Difx-users <
>> difx-users at listmgr.nrao.edu> wrote:
>> 
>> Dear DiFX community,
>> 
>> A few days ago I got an annoying bug in the difx2mark4. In my case I
>> have
>> station in the vex file with 2-symbol code in capitals (say 'JL'),
>> after
>> correlation I have run difx2mark4 and got an error in the root file
>> for
>> this
>> station. In occures that somebody converts 2-nd symbol of station
>> name to
>> lower case, what cause problem of different tags in the ovex file.
>> Patch
>> file
>> and the path to that file in difx2mark4 source code are attached.
>> 
>> Can somebody help to review patch and incorporate it to the
>> codebase? All
>> latest master_tag versions are affected, but there is only single
>> release
>> in
>> git, so, I think I need somebody who knows what to do.
>> 
>> regards,
>> Alexey
>> 
>> --
>> Best regards,
>> Alexey Melnikov, Scientific Researcher
>> Laboratory of Radio Astronomy Observations
>> The Institute of Applied Astronomy RAS
>> St.-Petersburg, Russia, tel. +7 (812) 2751106
>> 
>> Please avoid sending me Word or PowerPoint attachments.
>> See http://www.gnu.org/philosophy/no-word-attachments.html
>> _______________________________________________
>> Difx-users mailing list
>> Difx-users at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>> 
>> --
>> !=============================================================!
>> Prof. Adam Deller
>> Centre for Astrophysics & Supercomputing
>> Swinburne University of Technology
>> John St, Hawthorn VIC 3122 Australia
>> phone: +61 3 9214 5307
>> fax: +61 3 9214 8797
>> !=============================================================!
>> _______________________________________________
>> Difx-users mailing list
>> Difx-users at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/difx-users
> _______________________________________________
> Difx-users mailing list
> Difx-users at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/difx-users