[evlatests] Subreflector rotation statistics

Wed Apr 3 16:21:10 EDT 2019

A note on why none of the new ACUs show the bad behavior. When I brought this up at the division head’s meeting yesterday, Jon pointed out that new ACU antennas get FRM maintenance when the new ACU is installed. So they’re less likely to behave badly. 

       -Bryan

Sent from my iPhone.

> On Apr 3, 2019, at 14:15, Paul Demorest <pdemores at nrao.edu> wrote:
> 
> hi all,
> 
> This is a report on an analysis of VLA subreflector rotation times I recently did (some of you will have seen a version of this already; there is a little new info in here but no change in basic conclusions).  This was motivated by recent anecdotal reports from operators and analysts about specific antennas often being flagged due to subreflector rotation for much longer than expected, sometimes resulting in their missing calibrators, etc.  I thought it would be useful to take a more systematic look at recent data for problems like this.
> 
> First, the main conclusions are:
> 
> - There are several "bad" antennas that frequently spend >~10x the time flagged due to subreflector rotation as the rest, often for minutes at a time.  These are ea05, ea11, ea22, ea23, and ea25.  These should be prioritized for FRM maintenance if possible.
> 
> - There are a few more "marginal" ones that show similar behavior but not quite as severe (ea09, ea10, ea12, ea13, ea15).
> 
> - All the "bad" and "marginal" antennas have old ACUs.
> 
> - Not all old-ACU antennas act badly, for example ea03 and ea04 look generally pretty well-behaved.  But even these "good" examples spend typically ~50% more time flagged than new-ACU antennas.  So the new ACUs and associated mechanical overhaul are clearly an improvement (this is probably not news to many of you!).
> 
> More details about this analysis:
> 
> I gathered data on this from the SDMs currently available in the MCAF workspace.  Right now this goes back to the beginning of the year.  To avoid confusion from test/maint time, I only counted real science observations, identified as those datasets that start with '1' or 'V'.
> 
> For each day (MJD) I add up all the time each antenna is listed as being in the SUBREFLECTOR_ERROR state in Flag.xml.  This only counts rotation errors (I haven't looked at focus but could in the future).  Since there will be different numbers of band changes each day, I then divide all the times by the median of the 10 best (least flagged) antennas for that day.
> 
> For a second statistic, I also looked at the duration of each flag event.  For reference, a typical subreflector rotation for a band change should take somewhere between 5 and 25 seconds depending on which bands are in use; Rick took a close look at this recently, see his emails to this list in Nov 2018 titled "Band Change Times."  The assumption that band changes take ~20s is baked into our software in several places (OPT, observing scripts).  I counted up all the instances where an antenna was flagged for >30s or >120s, these will be potentially bad for observations.
> 
> Both of these metrics are plotted versus antenna number for a week's worth of data at a time (starting on Wednesday evenings).  The rotation time plot has one point per antenna per day for a week.  The flag duration counts are cumulative for the whole week.  See attached png showing the most recent week, and pdf showing all available data.
> 
> This analysis has an implicit assumption that all antennas are getting commanded to do the same thing.  This will occasionally not be true, for example if an antenna is removed from observing for part of a day for some reason.  So isolated data points away from 1.0, or small non-zero numbers of long-duration flags can probably be ignored.  But long-term patterns where certain antennas have consistently high/scattered points or many long-duration flags are meaningful, for example the "bad" ones I mentioned above.
> 
> The other situation that may confuse this analysis somewhat is subarray observations.  To help avoid this, I've excluded all datasets that used less than 24 antennas.  There may be some residual effect on the first full-array observation following a subarray project since the antennas will have different starting subreflector positions.  These have not been removed since they are more difficult to automatically identify.  But I think this happens infrequently enough that it's not a big problem.
> 
> Please let me know if you have comments, suggestions, or questions about any of this.
> 
> Cheers,
> Paul
> 
> 
> <58570.subrot.png>
> <subrot.all.pdf>
> _______________________________________________
> evlatests mailing list
> evlatests at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/evlatests