[evla-sw-discuss] Switch error May 24th

[Ken Sowinski]

On Mon, 27 May 2013, James Robnett wrote:

> We got another of those switch memory reference count errors on
> the main site switch at 15:25:06 MDT on May 24th (friday).
>
> I see no evidence it caused any down stream problems but it means
> the IOS upgrade didn't help.
>
> Can somebody describe what was going on with the instrument at
> that time ?  Anything (scan boundary, config change, etc) that might
> cause a burst of traffic.


15:25:06 MDT is 21:25:06 UTC.  At that time Vivek and I were trying
to understand spectral artifacts when using the stage 2 mixer of the
filter FPGA. The correlator setups were simple, not much data was
generated.

A script was started at 21:17:14 and stopped at 21:27:10.  Scans were
one minute long, it reconfigured the correlator every three minutes,
trying to find out exactly when would be much more work.  In any case
scan boundaries occurred at the 25 second tick, 15:24:25 for example.

Ken


> I'm more suspicious of rare OS bug than hardware fault. If that's the
> case it might be related to our traffic pattern,  i.e. bursty but
> synchronized traffic, particularly multicast packets.