[daip] AIPS on Supercomputer

[Eric Greisen]

On 03/30/2016 11:14 AM, Gary Burton wrote:
> Hi Eric,
>
> Sorry to bother you. I'm just installing AIPS on our Supercomputer at
> the University of Portsmouth. I have successfully run the install.pl
> <http://install.pl>. I was wondering if you could point me at any
> documentation on how to setup users for a multi users environment.
>
> Kind Regards
>
> Gary
>
> ps. Any other sysadmin docs would also be good.
>
> --
> Institute of Cosmology and Gravitation (ICG),
> Dennis Sciama Building, DS2.04
> Burnaby Road, Portsmouth, PO1 3FX
> Gary.Burton at port.ac.uk <mailto:Gary.Burton at port.ac.uk>
> +44 (0)23 9284 5157 (office)
> +44 (0)7967337957 (mobile)
> web: http://www.sciama.icg.port.ac.uk <http://www.sciama.icg.port.ac.uk/>
>


There is an AIPS Manager FAQ page
  http://www.aips.nrao.edu/aipsmgr/index.html
which should answer some questions - the multiple computers question is 
quite a ways down the page about "configurations".  Note that, if you 
are sharing data areas, each user should use a single aips number (or an 
agreed upon set of numbers).  In this way data are kept separate.  I 
also attach a memo about yet another "dadevs" file that users can employ 
to describe their data areas.  Typically now users can set up their data 
areas on the lustre disks of our central server and then force them to 
be included with $HOME/.dadevs.always.  I hope that the FAQ and this 
note help but please ask more detailed questions as needed.

Regards

Eric Greisen

-------------- next part --------------

            Using lustre disks with AIPS
            ----------------------------

The current plan is to assign users their own data area on the lustre
system named /lustre/aoc/users/uname.  Unlike the assumptions made by
AIPS, this is a data area which is associated with a user rather than
with a computer.  You will probably wish to use this area from
whatever computer you are currently using - your desktop, one of the
large public machines (nm-post-1, nm-post-2), or even machines on a
cluster.  To allow this, we have created additional logic in AIPS
handling of data areas.  Most users obtain their data areas via an
NRAO maintained file $AIPS_ROOT/DA00/DADEVS.LIST.COAOARN and, while on
machine XXX, they get to use any area listed in that file with XXX in
its name.  But, when you run on another computer, say YYY, you get the
data areas for YYY unless you say da=XXX on the command line.  That
gives you both the YYY and XXX data areas.

To invoke the new logic, create a file in your home area named
.dadevs.always.  List in that file any data areas you wish to use on
any computer - usually your lustre aips data areas.  The format of the
file is fussy - the first character in the line is a plus sign or a
minus sign.  Then there are 2 blank characters and then the full path
name.  Data areas with a + sign will come first in your disk list,
those with a - sign will come later, after the current host's disks.

Disk 1 is special in AIPS since it contains the SAVE/GET files, the
TGET file, the message file, and more.  It may be convenient to have
a lustre data area so that you get to have the same SAVE/GET etc no
matter what compute server you are using.  HOWEVER, we may be forced
to institute a policy of automatically deleting unused files on lustre
when we eventually encounter disk space problems.  As a consequence,
you should think twice about using a lustre area for long-term
storage.  It is possible that a disk space clean program could be
invented which will not delete certain types of files such as TGET,
SAVE/GET, and the like.

Because words like those above are confusing, I attach the commands I
typed to produce a working lustre environment for myself:

cd /lustre/aoc/users/egreisen
mkdir AIPS
chgrp aipsuser AIPS
chmod g+ws AIPS
cd AIPS
mkdir LUSTRE_1 LUSTRE_2
touch LUSTRE_1/SPACE LUSTRE_2/SPACE
cd
emacs .dadevs.always
+  /lustre/aoc/users/egreisen/AIPS/LUSTRE_1
-  /lustre/aoc/users/egreisen/AIPS/LUSTRE_2

and then I saved the file and exited.

When I issue the command

aips da=megrez

I get the disks

   Disk 1 (1) is /lustre/aoc/users/egreisen/AIPS/LUSTRE_1
   Disk 2 (2) is /DATA/PRIMATE_1
   Disk 3 (3) is /DATA/PRIMATE_2
   Disk 4 (4) is /DATA/PRIMATE_3
   Disk 5 (5) is /DATA/PRIMATE_4
   Disk 6 (6) is /DATA/PRIMATE_5
   Disk 7 (7) is /lustre/aoc/users/egreisen/AIPS/LUSTRE_2
   Disk 8 (8) is /DATA/MEGREZ_1
   Disk 9 (9) is /DATA/MEGREZ_2


If you have questions or want help, see me.

Eric Greisen