[Difx-users] DiFX in the Cloud - Status

[Geoff Crew]

Hi Stuart,

The differing integration periods alone would be sufficient
for slightly different results on the observables.  However
I note that fourfit was commanded differently (a control file
in one case [claiming manual phase cals] and none in the other
[defaulting to multitone]), that the phase cals are different
(which is perhaps a bit worrisome), and that this is an old
version of fourfit.

I think you'll get a better comparison if you use the version
of the hops tools currently on the trunk (and awaiting tagging
with DiFX 2.5).  Or maybe you can save the data sets and try
again once DiFX 2.5 is released?

-- 

		Geoff (gbc at haystack.mit.edu)

On Wed, Nov 23, 2016 at 10:42:12PM +0000, Stuart Weston wrote:
>    Dear DiFX users,
> 
>    Since my remote presentation at the Users & Developers Meeting earlier
>    this month we have progressed to having a working install of DiFX in a
>    commercial cloud environment with the generous and kind assistance of
>    Catalyst NZ Ltd.
> 
>    I have started with a 2 antenna experiment we did last year as a fringe
>    test between Wa and Hh in C-Band. 16x16MHz bands, 2bit Mk5B recording so
>    1Gbps record rate.
> 
>    On our dedicated DiFX physical server (wark167 - 2 x Core i7 6 Cores
>    Hyperthreaded, 64GB RAM, RAID0 for data) this takes ~ 38 mins to correlate
>    with np=6 (see I hope attached fourfit plot file : wark167_no0006.pdf).
> 
>    I was able to use our "tsunami" and "jive5ab" tools to copy data into the
>    cloud storage. REANNZ our Research Network provider have provided a 1Gbps
>    connection between the themselves and Catalyst. They are in the process of
>    planning a 10Gbps link into Catalyst for other researchers who want to
>    make use of Cloud Computing.
> 
>    In the cloud VM (c1.c8r4 - 8 CPU's and 4GB RAM). I have split things into
>    three separate volumes to try and get the best I/O:
> 
>    /difx-data/ : This has the vex files, filelists etc, also the output goes
>    here
>    /vlbi-data/hhwa03/hh  - Hart raw data files
>    /vlbi-data02/hhwa03/wa - Wark raw data files
> 
>    Note it is easy to share these volumes with other instances.
> 
>    Using the same data files and settings (it was a straight copy from the
>    wark167 server, just changing paths to datafiles etc) with np=6 it takes ~
>    51mins to correlate (see I hope attached fourfit plot file :
>    lwcc_no0006.pdf). Granted this VM is not comparable to our server above
>    but this was what Catalyst gave me to get started (again we don't want to
>    upset their paying customers)
> 
>    Over the coming weeks I will be working with Catalyst to try and improve
>    those correlation times. We will also benchmark the same LBA session I
>    presented our initial work with at the meeting and IVTW2016. They just
>    have to do some housekeeping to give me > 10TB of volume storage and not
>    upset their paying customers.
> 
>    I am not sure if the fourfit plots attached will come through (perhaps
>    some one can suggest a better way to share them). But I note slight
>    differences between the two environments, for example:
> 
>    Item        wark167              Cloud
>    SNR        253.1                   253.2
>    Int time   297.886               297.720
>    Amp        10.726                10.734
>    Phase       -88.4                  -87.5
>    ......
> 
>    Interesting the start times are slightly different ?
> 
>    Start       133601.57            133602.10
>    End        134059.90            134059.90
> 
>    The Phase Amplitude plots look the same
> 
>    Catalyst are aware of their data storage charges being high, they are open
>    to the idea of hosting 3rd party eqpt so we could put a large storage
>    array in for ourselves. But they are also looking at bringing those
>    charges down. They have some nice scripting tools that mean the head node
>    could parse the VEX file and spin up as many worker instances as required
>    (quickly !), something I will explore with them in due course. We can also
>    explore having several head nodes and running several across sub-sets of
>    the data files to get better turn around.
> 
>    I hope this is of interest to the DiFX community.
>    Stuart Weston Bsc (Hons), MPhil (Hons), MInstP
> 
>    Mobile: 021 713062
> 
>    Skype: stuart.d.weston
>    Email:  stuart.weston at aut.ac.nz
> 
>    http://www.atnf.csiro.au/people/Stuart.Weston/index.html
> 
>    Software Engineer
>    Institute for Radio Astronomy & Space Research (IRASR)
>    School of Engineering, Computer and Mathematical Sciences
>    Faculty of Design and Creative Technologies
>    Auckland University of Technology, New Zealand.
> 
>    http://www.irasr.aut.ac.nz/
>    !DSPAM:58361c2853811550310194!



> _______________________________________________
> Difx-users mailing list
> Difx-users at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/difx-users
> 
> 
> !DSPAM:58361c2853811550310194!