[daip] AIPS Site Setup

[Patrick P. Murphy]

On Thu, 18 Oct 2001 13:25:37 -0700 (PDT), Terry Hancock
   <hancock at ipac.caltech.edu> said: 

TH> I've been given the task of setting up AIPS along with a variety of
TH> other tools at IPAC (Caltech/JPL). I'm pretty new to AIPS, so I'm not
TH> really familiar with the history of the program or its complete
TH> structure, but there appear to be some serious violations of the
TH> "software application" model that I'm accustomed to on Unix systems,

As Eric pointed out, AIPS was developed before Unix was in widespread use.
On a Modcomp, I believe.  It migrated from there to VAX/VMS systems in the
early 1980's, to Unix (on an IBM 360 using VM and a guest Amdahl version
of Unix), thence to Convex, Cray, Alliant, Sun, Ultrix, AIX, HP-UX,
SG-Irix and eventually Linux.

TH> 1) The AIPS install wants a list of all hostnames that will be used
TH>    with it. Furthermore, the hostnames in HOSTS.LIST (a previous
TH>    installation) are in all-caps,

Yes.  The scripts pipe the actual names through tr '[a-z]' '[A-Z]' for
historic reasons (when we had to support the caseless and uppercase
preferring VMS operating system alongside Unix).

TH>    Is it possible to create a single "LOCALHOST" entry and allow any
TH>    computer on the net (Since localhost = 127.0.0.1 always resolves to
TH>    the current computer under Unix)?

No.  That's because each machine needs its own set of system files.  These
are stored in the DA00/$HOST/ subdirectories (where $HOST expands to the
[uppercased] hostname.  The accounting, batch system, tape locking and
many other things would get really screwed up if you tried to share this
area among different hosts; trust me, your users would not be pleased.

Also, this would completely break the ability to do this sort of thing:

      1) fire up AIPS locally on your desktop, get the "TV" (image
         display) window up and running.
      2) slogin great-big-whopping-compute-server
      3) on great-big-whopping-compute-server, fire up aips thus:
           aips tv=myhost tvok

This scenario is one where the AIPS program and tasks operate on a big
remote compute server, but use the SSSIN protocol (see install notes;
/etc/services or NIS services map needs some entries) is used by these
tasks to communicate directly with the running "TV" on the local client.
Interactive performance substantially increases over the alternative of
sending the "TV" running on great-big-whopping-compute-server over X11.
   
TH> 2) AIPS wants to maintain user data directories under the AIPS
TH>    $AIPS_ROOT directory.

No, not really.  We (NRAO, and ATNF whence we inherited the current model
in 1992) have our own paradigm of how data areas are laid out, involving
a dedicated automounter map called "auto.DATA" that maps all data areas
everywhere under a /DATA/ root.  As this choice is not for everyone, there
are two ways to get around it:

    1) put symbolic links in the $AIPS_ROOT/DATA/ area pointing at the
       real data areas; or
    2) bypass the $AIPS_ROOT/DATA/ directory altogether this and put the
       actual data areas directly in both DADEVS.LIST and NETSP (these
       files are in $AIPS_ROOT/DA00/).

The second approach works well, provided you are aware of the logic used
in the DADEVS.PL script (called as part of the standard AIPS startup):

    1) include any data areas that have a "+" in column one in DADEVS.LIST
    2) include any data areas matching ${HOST}$ (note end of line regexp);
    3) include any data areas matching ${HOST}_ (underscore after hostname);
    4) add any data areas specified via the DA=... aips command line option.

There more to that script, including several alternative files (site
and/or user specific) but this gives the gist of it.  My point here is
that naming of the actual data directories is significant.  Where they
reside is not (assuming you're not concerned with network performance
issues).

TH>    On this network, that area is intended to be mounted read-only,

The following files need to be user-writable for AIPS to work:

    - all actual user data areas
    - the $AIPS_ROOT/DA00/$HOST/ directories
    - the contents of $AIPS_VERSION/$ARCH/MEMORY/

You may have to play some tricks via symlinks if you want the actual
$AIPS_ROOT area to be mounted "ro".

TH>    Can I use an environment variable based path for users?
TH>    I.e. $HOME/.aips/ instead of an absolute path.

I think what you're getting at here is implemented in AIPS via the
personal ~/.dadevs files.  These are useful but do imply a shifting of
responsibility from the AIPS and/or system administrator to the user in
terms of maintenance.  The user has to keep this current (and they don't;
trust me, I've been helping AIPS users here at NRAO and all over the world
since 1991 and earlier).  If they just rely on the system based
$AIPS_ROOT/DA00/DADEVS.LIST (or DADEVS.LIST.${SITE} if it exists) then
they won't get burned when shared data areas are shuffled, added or
deleted.  I've seen data areas defined in a ~/.dadevs file that pointed to
a disk that hit the dumpster 5 years previous :-(

TH> I would like this set up so that any user on any host that can mount
TH> the partition AIPS is loaded on can use the system without a
TH> centralized installation.

The only way to make this work is to add all hosts into the HOSTS.LIST
file, then run the $SYSUNIX/SYSETUP script to create the system areas for
all of them.  Yes this is clunky.  Yes, it could be changed; want to
volunteer to do the work?  :-)  (There's a good reason we haven't done
this ourselves: it's a heck of a lot of work).

TH> The standard Unix approach is to keep program and
TH> user areas clearly separated

The problem AIPS has here is that it requires host-specific system files
for each system.  This is in part driven by the legacy of AIPS (remember
how much memory one had to work with in 1980?), by the tight linking of
the standard image display to both an INET-based socket and to state
information on the display, and stuff like that.

I went over all the points you've raised with another site manager at a
major University about 8 years ago.  The problem boiled down to a
philosophical argument: collaboration and shared resources (the NRAO
model) versus cubicle-like isolation where nobody shares data areas,
ever.  I ended up telling that manager we couldn't help him because he
insisted he had to drastically modify the system to make it work the way
he wanted.  

TH> Program areas are generally read-only for security reasons.

The exceptions I list above were that way for historic reasons, and
because AIPS is a nearly-all-Fortran application.  Originally the I/O was
done via Fortran, and the f77 standard lacks a "READONLY" attribute on the
OPEN() call.  That's less so now (most AIPS I/O is done via the C language
"Z" routines).

TH> Given that AIPS is deployed mostly on Unix networks
TH> nowadays, I would think that you've probably dealt with
TH> these kinds of issues in the past. I'd be interested
TH> to know what sort of solutions have been found for them.

Solutions in this case imply resources, something we've been lacking (both
in terms of people and time).  In almost all cases, the ideas you've
suggested have in fact been considered, and we had to pass on them all
because there aren't enough people left in the (Classic) AIPS group.

I'm no longer in the AIPS group formally, but I do help out from time to
time with questions such as yours.  I hope this somewhat verbose reply
helps clear up some of the issues you've raised.

				- Pat
-- 
 Patrick P. Murphy, Ph.D.              Division Head, CV Computing, NRAO
 Home: http://www.chien-noir.com/    Work: http://www.nrao.edu/~pmurphy/
 Spam poison: Argos at backspacedk.com  borax at porkerl.gov  nead at glaresg.edu
 "Linux is Inevitable."  "Why?" "Because it's alive!" - John MadDog Hall