[evlatests] Dead Scans!

[Rick Perley]

    I reported earlier on a phenomenon whereby neither of the EVLA antennas
took valid data for an entire scan, while all VLA antennas continued as 
normal. 
Furthermore, this problem is almost certainly not due to antenna 
malfunction,
because:
    a) It always takes out both antennas at once
    b) There is no preference for any particular band
    c) The bands taken out are known to be working from observations
taken immediately before or after. 

    I have now two more datasets, the first taken Thursday afternoon, the
second overnight Thursday/Friday.  Both show the dead scan phenomenon. 
The data are taken in single-IF mode, spectral line (the intention being to
firmly establish the antenna sensitivities).  Data were taken with 5-second
integration, 128 channels (6.25 MHz BW).  All antennas and IFs were
observed sequentially -- 20 observations per cycle.  (That's a lot of data!)
The duration per observation was 1.5 minutes. 
********************************************************************
    a) Thursday afternoon.  IAT time from 18:41 to 20:41. 

    The first 5 scans are dead due to a self-inflicted error.  We won't 
talk about these.
(Note to self:  the EVLA observe files must have the correct day number!)

    Of the next 15 observations, 2 were dead for both antennas 14 and 16. 
The Band-IFs afflicted were  L-D and K-A.  There were 7 good scans
in between the two dead ones.  No pattern can be discerned from so little
data.   The dead scans were at 19:07 and 19:39 IAT.  The system flagged
the latter (at K-band), but not the former! )
   
    The final 8 scans are *are* dead for antennas 14 and 16 (but good for
all VLA data).  Clearly, something died.  But there is no record in the log
other than the following note:
-----------------------------------------------------------------------------------------
Ken says EVLA antennas quit receiving commands, although executor
continues to run; we will proceed with completion of Rick's file and then
try starting Vivek's immediately after and see what happens.
------------------------------------------------------------------------------------
    The approximate time at which the EVLA antennas stopped working
is 20:12. 
**************************************************************
    b) Overnight Thursday/Friday.    IAT time from 4:38 to 8:00.

    I ran the same file (with a different source, of course!).  The long 
duration
permitted cycling around 2 1/2 times, giving me 51 separate observations. 

    Six of these 51 observations show the phenomenon.  The band-IFs
affected were:  Q-A, K-B, L-B, Q-C, X-D, K-D.  The sequence of
good-bad-good ... scans was:
       17-1-6-1-3-1-5-1-3-1-6-1-5
    (i.e., 17 good ones, 1 bad one, 6 good ones, 1 bad one, 3 good ...)

    I can discern no pattern here (in contradiction to yeterday's
speculated pattern). 
    In all cases, both EVLA antennas are completely dead for the
whole scan. 

    I filled the data twice, once with all flags off, and once with
the flags on.  Both K-band dead-scan events were flagged.  The
L, X, and Q-band events were not flagged. 

    Ken points out that the only EVLA system parameters that the VLA
flagger is monitoring is total power.  Hence, he proposes that the
reason the dead K-band observations are flagged, and the others not,
is that the power difference between Kband and the preceding observation
is large, while it is not for L, X, and Q.  The order
of observations is always:  L -> C -> X -> K -> Q -> L ... so K-band
is always preceded by X-band. 

    Ken wonders if the executor commands for these dead scans are
getting lost, so the antennas remain at the previous band.  In that case,
we might see a weak residual signal (subreflector remains at the prior
band for both antennas, but the phase rotation will be incorrect). 
Unhappily, the prior observation is always on the noise source, so this
prediction can't be tested. 

    Is there some monitor record of what band/source the antennas
are actually at (rather than what we told them to do?) 



***************************************************************

 
    I'll report later on data quality, sensitivity, phase stability, etc.