[evla-sw-discuss] Power outage tomorrow

[James Robnett]

Just so we're moderately on the same page and can ensure everything
is down for Bob by 8am.  This is largely for my group but so everyone
else knows.

Observing is supposed to finish by 7am.
Kevin will be at the AOC around 7am,  Kscott and I sometime shortly
thereafter.

Soon after 7am Operators tell Kevin it's safe to shut down correlator.

7:30 Kscott checks with Operators that correlator is down.  Tertiary
servers (backups etc), then secondary (M&C, CBE etc) then primary
(Oracle, Lustre, CBE control) servers shutdown in order.

7:50 or so Kscott lets Operators know servers are down and power can
be cut whenever.

11am to whenever Jeff Long, Bob Huber and I will be in correlator room
  o moving WIDAR/CBE network cables to solve frame delay issues,
  o moving Lustre to make way for full CBE and adding infiniband cards
  o Installing new 120v PDUs in existing racks to better distribute power

3pm or whenever power restored to then + 45 minutes

Jeff and I will make sure everything is up,  we'll let operators and
evla-sw group know when the servers are up.  Please don't randomly
restart software (executor,telcal) because they're not working till
we say everything's up.  That sometimes causes more trouble than it
solves.

Before we leave the site we'll check with Ken/Barry/Whomever and
see that things seem to be working at least to the extent that they
can be kicked remotely.

James