[evlatests] Set-and-remember failures at L and S bands ...

[Rick Perley]

    The flux density run has triggered a failure mode in the 
'set-and-remember' attenuation setting regimen, seen only at L and S 
bands.  A number of antennas are involved, and no clear origin is 
obvious (to me, at least). 

    At these two bands, very little change in total noise power should 
be seen as a function of elevation or time, so we would expect that 
independent of whether the required first scan was long enough for the 
power settings to be remembered, the power levels reported by PSum (and 
PDif) should not change appreciably -- certainly no more than 1dB. 

    Nearly all the events reported below involve the power  jumping up 
by more than an order of magnitude.  This cannot be caused by an RFI 
source being seen at the wrong time, as this would cause the attenuators 
to be set too high.  So some other cause must be found. 

    A short log of events.  Unless otherwise noted, all of these are for 
L-band. 

    1)  ea01, on all four IFs, jumped up by a factor of about 20 in the 
middle of the make-up four-hour observation (January 20).  There is a 
missing observation prior to the one which first showed the excessive 
power, and the visibility fringes are of poor quality following the 
event.   This high state was maintained until the end of the run. 

    2) ea03 set itself at a power level at least of forty too high at 
the beginning of the run (January 19, at 00:20 IAT), on all four IFs -- 
which included a scan long enough for a safe set-and-remember 
observation.  This state remained until the first system reset, after 
which the power level was correct.

    3) ea09, on the AC IFs only, set itself at a level 20 times too high 
-- both at the beginning of the run (R and L), and at the beginning of 
the make-up run (RCP only).  What distinguishes this from the others is 
that both of these followed at X-band 'dummy' setup card.  I'm betting 
that this antenna, and possibly ea03, were using the attenuator settings 
from X-band...

    4) ea15, for IF 'A', set up at a level 40 times too high for the 
initial scan -- the same situation as for ea03 and ea09. 

    5) ea17 is a very special case, as it was the only situation where 
the power went down at L-band.  At approximately 18h IAT (on January 19) 
-- and not following a restart after the four-hour gap that morning, the 
power levels dropped dramatically -- by factors of a few for IFs B, C, 
and D, and by a factor exceeding 20 for IF 'A'.  This event is not 
connected to that which threw off the power levels at C-band -- that 
event was at 20:45 IAT. 
    At S-band, the same event was noted, but here the power went *up*.  
For both L and S bands, the quality of the PDif was very much degraded 
during the extent of the event. 

    6) ea25 jumped up in power at both L and S bands, also starting at 
the same time as noted above, by at least two orders of magnitude!  The 
problem was absent at the subsequent makeup run (which started 6 hours 
after the completion of the scheduled run).   The same characteristics 
were seen at S-band.  Both both bands, the quality of the PDif data was 
greatly (hugely!) degraded. 

    7) ea26 gave nonsensical PDif data, starting at 14:40 on the makeup 
run, January 20.  This event is similar to the problem on ea01 (#1, 
above), but not coincident in time. 

    In summary:  the system is all too commonly either not setting up 
at, or not staying at, reasonable power levels at L band, and on 
occasion, S-band.  This problems do not appear to affect the other 
bands.   No clear origin for the problem is indicated.