[evla-sw-discuss] EVLA power outage

[VLA Operations]

I'm not aware of any further computer issues at the moment, but we're not at the 
stage of trying to run any test scripts.

Widar just finished powering up.  Kerry is working on it from the AOC and will 
get the D352s running, which will take at least another hour.

Rob Long is at the VLA getting the CER LO/IF units (L252s, P350, etc) powered 
up, and once those are okay will start the process of syncing every antenna.

Rudy Latasa has most of the Cryo systems under control and is working on a 
couple receiver issues.
Servo systems have been fixed.

Michael Rupen and Joe McMullin were emailed about the situation, and I will talk 
to them shortly.

Note: We are on utility power, however the generators will not come on if we 
lose power again.

Cheers,
Matt

James Robnett wrote:
>   Apparently there was a VLA power outage last night.  Kscott and Matt
> have most everything physically up but not necessarily logically up.
> 
>   There were definitely problems bringing things up so the correlator
> is probably not in a sane state.  The online system state is probably
> fairly undefined as well.  Timing on MIB's may or may not be accurate,
> we should verify that since the NTP servers take a while to sync.
> 
>   We need to enumerate the things that aren't working (if any) and fix
> them.  At the very least we need to make sure people who need to know
> actually know ... I've seen no email on it so I have no clue if others
> are aware of the potential for problems and are already dealing with it
> (modulo Kscott getting things physically rebooted).
> 
> James
> 
>