[evlatests] Failures during last night's tests

[Bryan Butler]

the references to the times when the operator gets the message about 
missing scans are only because my cron job to try to detect them only 
runs every 5 minutes.  that is the granularity of the messages.  there 
are alerts which we could pull out of the alert database (sent by MCAF) 
which would be more informative, but i don't think it's worth the 
effort, since we know what scans were missed now already.

i think this is all attributable to the CBE not getting configured 
properly - both the zeroes and the missing scans.  we should test, going 
back to slightly longer scans.

	-bryan


Rick Perley wrote, On 1/5/12 10:33 AM:
>      Another 'flux densities' test last night suffered multiple
> failures.  Although similar to those from two nights ago, there are
> significant differences.
>
>      I attempted to observe six sources, in this order:
>
>      3C84, 3C138, 3C147, 3C454.3, Uranus, J0006-0623.
>
>      For each source, the following sequence of bands was made:
>
>      L, XRefPnt, X, C, S, U, K, Q.
>
>      Except for the referenced pointing, the times were arranged to
> provide between 15 and 20 seconds of good data.  Typically, this means
> each band is observed for 20 to 35 seconds (the difference being due to
> how long it takes to reconfigure the correlator and/or move the
> subreflector).
>
>      The major fault (from the observer's point of view) was missing scans:
>
>      3C84 missed Ku, K, Q, and Ka bands.
>      3C138 missed Ku band.
>      3C454.3 missed S, Ku, K, Q, and Ka bands.
>      The last two sources provided no data at all.
>
>      The problem which affected the data from two days ago -- zero
> amplitude and zero weight data on approximately half of the antennas --
> did *not* occur in last night's test.  The only issue (from the
> observer's perspective) is the missing scans.
>
>      The operator's message, sent following the run, is reproduced
> below.  The first alert he references is 44 seconds following the last
> valid data (2:55:36).  Presumably, these missing data are all 'pure
> zeros', and hence are stripped out upon filling.
>      The operator's message also references emails about missing BDFs --
> he lists three messages, but there are five missing scans.  The times
> noted are a poor match to the missing scans:  at 2:35, we were getting
> good data at X-band on 3C138; at 2:40, we were doing a referenced
> pointing run on 3C147 -- which gave excellent solutions, and at 3:00, we
> were on or near Uranus (hard to know for sure, since no data showed up).
>
>
>
> -------- Original Message --------
> Subject: 	CBE config failure during TCAL0005
> Date: 	Wed, 04 Jan 2012 20:20:23 -0700
> From: 	EVLA Operations<observe at nrao.edu>
> Organization: 	National Radio Astronomy Observatory
> To: 	Michael Rupen<mrupen at aoc.nrao.edu>, Bryan Butler
> <bbutler at nrao.edu>, Rick Perley<rperley at nrao.edu>, Ken Sowinski
> <ksowinsk at nrao.edu>
>
>
>
> Hello.  The following alert appeared at 2:56:20, 2:58:23, 3:03:57,
> 3:05:56, 3:10:45, and 3:14:27 UTC:
>
> mccccm timerExpiredFor:TCAL0005.sb7289738 config-failure-cbe
>
> Also received 3 missing BDF e-mails at 2:35, 2:40, and 3:00 UTC.
>
> After TCAL0005 ended I ran the clearCorrelator script and then X_Osro.
>
> The CM screen for X_osro displays Failed for the CBE configure status.
>
> Dave
> EVLA Ops
>
> _______________________________________________
> evlatests mailing list
> evlatests at listmgr.cv.nrao.edu
> http://listmgr.cv.nrao.edu/mailman/listinfo/evlatests