<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-NZ" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi All,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I am still working away on DiFX in the Cloud.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I have tried ext3/4 and RAID0 in cloud block storage and it seems cloud I/O is a (as expected) limiting factor on correlation speedup.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">I have run a LBA session (6 antennas, 15 baselines) in the cloud and got upto 0.8 speedup using block storage in a RAID0 (one RAID0 for each antenna on a different Hypervisor).<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:#1F497D">I have no first hand experience with object storage, but I am being told by the Cloud Provider (Catalyst IT) here that they are faster than block storage. They tell me that Amazon Web Services (AWS) is using
Object Storage, it has API’s (but only in Python) and is S3 compliant.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><a href="https://www.catalyst.net.nz/catalyst-cloud/iaas/object-storage">https://www.catalyst.net.nz/catalyst-cloud/iaas/object-storage</a></span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Catalyst are asking me how easy it would be to build Object Storage addressing into DiFX ? I am thinking this is very much a non-trivial task and would need changes to many file access code programs for each data format.<span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">We are also looking at using “Ansible” (<a href="https://www.ansible.com/cloud">https://www.ansible.com/cloud</a>) to break a job out and create a head and worker nodes on each hypervisor, so each exp.input gets
processed in a different group where a group consists off Head + N*Workers. “Ansible” will dynamically build the VM’s (Head + N*Workers groups, one group per hypervisior) submit the mpirun job on each head (see attached for the tree structure of VM’s). Once
finished break it all down so you only have the VM’s up and running for how long the job takes to run.<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We haven’t used “Ansible” in anger yet on an actual correlation, so far just to test provisioning VM’s taking the input file and the expresso cluster definition file. We still have physical limits ie 15 hypervisors so we can create entries
in the cluster definition file taking that limit into consideration. We then provision the VM’s required on-demand. I hope by the end of this week to have used “Ansible” in anger.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black;mso-fareast-language:EN-NZ">Stuart Weston
</span><span style="color:black;mso-fareast-language:EN-NZ"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black;mso-fareast-language:EN-NZ">Mobile: 021 713062</span><span style="color:black;mso-fareast-language:EN-NZ"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black;mso-fareast-language:EN-NZ">Skype: stuart.d.weston<br>
Email: <a href="mailto:stuart.weston@aut.ac.nz" target="_blank"><span style="color:blue">stuart.weston@aut.ac.nz</span></a></span><span style="color:black;mso-fareast-language:EN-NZ"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:black;mso-fareast-language:EN-NZ"><a href="http://www.atnf.csiro.au/people/Stuart.Weston/index.html" target="_blank"><span style="color:blue">http://www.atnf.csiro.au/people/Stuart.Weston/index.html</span></a><br>
<br>
Software Engineer<br>
Institute for Radio Astronomy & Space Research (IRASR)<br>
School of Computing & Mathematical Sciences<br>
Faculty of Creative Technologies<br>
Auckland University of Technology, New Zealand.</span><span style="color:black;mso-fareast-language:EN-NZ"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black;mso-fareast-language:EN-NZ"><a href="http://www.irasr.aut.ac.nz/" target="_blank"><span style="color:blue">http://www.irasr.aut.ac.nz/</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black;mso-fareast-language:EN-NZ"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black;mso-fareast-language:EN-NZ"><img border="0" width="622" height="245" id="Picture_x0020_1" src="cid:image002.jpg@01D26CB8.0319A400" alt="NewIRASRLogo"></span><span style="mso-fareast-language:EN-NZ"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-NZ"><o:p> </o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>