[evlatests] phase noise on outer antennas {External}

Ken Sowinski ksowinsk at nrao.edu
Thu Nov 14 10:34:45 EST 2024 

I  await the results from the recent test.  Comments inline.

On Thu, 14 Nov 2024, Paul Demorest via evlatests wrote:
> Thanks, I got some more 10ms data today to verify, will report back 
> tomorrow.
>
> In the meantime, a week or so ago, I recorded raw data from four 
> antennas (ea11, 13, 18, 22) to get a look at higher time resolution. 
> Two inner and two outer antennas, and the only "bad" one in this set is 
> ea18.  I processed this at 0.5 ms to be able to see fluctuation 
> frequencies up to 1 kHz.  Attached plot shows a power spectrum of the 
> phase noise on all 6 baselines, with 0.1 Hz resolution.  A few notable 
> things:
>
> 1. The ea18 baselines all show the excess phase noise, as seen before.
>    The amplitude starts dropping around 50 Hz and gets down to a
>    comparable level to the good baselines around 200 Hz.  I think this
>    is all expected given a PLL bandwidth of 50 Hz previously noted in
>    this thread.
>
> 2. In addition to the broad-band noise there are some notable spikes in
>    the ea18 data at 19.2, 26.2, 29.7, 39.3, and 59.5 Hz.  Aside from

19.2 Hz is the old wavegiuide transmit/receive cycle frequency.  There
may still be some remnants of it around in the master lo.  The others
mean nothing to me.  If 59.5 Hz is related to mains power, I am surprised
it is so far from 60 Hz.  Was this during an SEC outage while using the
generator?

>    that last one being near 60 Hz power-line freq, none of these numbers
>    are especially meaningful to me.  Do they ring a bell for any of you?
>
> 3. The "good" baselines show a few spikes as well, at 4.0, 10.0 and 20.0
>    Hz.  Though at a much lower level so not really a problem more of a
>    curiosity.  I believe 4 Hz is a known correlator artifact related to
>    the length of a memory buffer (Ken can probably confirm).  And 10 Hz

The delay buffer is long enough to hold 250 msec of data, but I am hard
pressed to see how this would introduce a periodic signal in the sampled
data.

>    and harmonics are common throughout the VLA for multiple reasons, and
>    have been noted many times before.
>
> Cheers,
> Paul
>
> ________________________________________
> From: evlatests <evlatests-bounces at listmgr.nrao.edu> on behalf of Leon Abeyta via evlatests <evlatests at listmgr.nrao.edu>
> Sent: Wednesday, November 13, 2024 2:13 PM
> To: evlatests at listmgr.nrao.edu
> Cc: Jeffrey Yarnell; Michael LeBlanc
> Subject: Re: [evlatests] phase noise on outer antennas {External}
>
> The patch panel for the 3 farthest antennas were checked today
> (11/13/24) and they were all without attenuators. Attenuators labelled 2
> dB were added to the 3 antennas in the hope that this will improve the
> phase noise.
>
> Leon Abeyta
>
>
> On 11/7/2024 9:12 AM, Ken Sowinski via evlatests wrote:
>> On Tue, 5 Nov 2024, Jim Jackson via evlatests wrote:
>>> Thanks Paul.
>>>
>>> Usually this has been accomplished by adding an attenuator in the
>>> optical path.  The easiest place to do that is in the patch panel
>>> rack in the Control Building right after the L353 transmitters on the
>>> fiber going out to the antenna.
>>
>> This suggests that it is happening in the long fiber run or the detetor
>> at the antenna; not the optical transmitter.  Would you learn anything
>> by comparing the returned signal on the RT phase fiber with the optical
>> output of the L353?
>>
>> Ken
>>
>> ]
>>> The effect Rob and I analyzed years ago was clearly related to the
>>> optical power level at the input jack of the optical receiver in the
>>> L304.  I'll check with Leon to confirm exactly what they did.  I
>>> suspect they added optical attenuation at the patch panel in the
>>> building to get the measured optical power in the L304 within the -17
>>> to -20 dBm spec, then reduced the RF attenuation between the L304 and
>>> L305 in the antenna to get the RF power into the L305 at -26.5 dBm
>>> where it likes to be.
>>>
>>> Cheers,
>>> Jim
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: Paul Demorest <pdemores at nrao.edu>
>>> Sent: Tuesday, November 5, 2024 4:27 PM
>>> To: Barry Clark <bclark at nrao.edu>; evlatests <evlatests at nrao.edu>;
>>> Jim Jackson <jjackson at nrao.edu>
>>> Subject: Re: [evlatests] phase noise on outer antennas {External}
>>>
>>> Thanks Jim.  A couple follow-ups:
>>>
>>> I did the test suggested by Ken of comparing a far-North source
>>> (J1800+7828) with one near the equator (3C273 aka J1229+0203). The
>>> fringe rate due to earth rotation decreases by about a factor of 5
>>> (proportional to cos(dec)) between these two.  No difference was
>>> observed in the amplitude or appearance of the outer antenna phase
>>> jitter; everything looks about the same.  I think this continues to
>>> suggest that this is not a correlator-based effect.
>>>
>>> I think Barry's suggestion that this effect might occur at the
>>> transmit side of the LO fiber makes a lot of sense given what we've
>>> observed so far.  When the ea26 power level was reduced was that a
>>> change at the transmitter, or was an attenuator installed at the
>>> antenna?
>>>
>>> Also the ea18 optical power is very slightly outside your recommended
>>> range: it's -16.2 dBm, which is now the highest value on the array
>>> after the ea26 change.  However the other two arm-end antennas are
>>> much lower (ea21=-18.3 and ea24=-19.4).
>>>
>>> Cheers,
>>> Paul
>>>
>>> ________________________________________
>>> From: evlatests <evlatests-bounces at listmgr.nrao.edu> on behalf of Jim
>>> Jackson via evlatests <evlatests at listmgr.nrao.edu>
>>> Sent: Sunday, November 3, 2024 12:03 PM
>>> To: Barry Clark; evlatests
>>> Subject: Re: [evlatests] phase noise on outer antennas {External}
>>>
>>> I spoke to Leon about this last week.  We know that the Fiber Span
>>> optical receivers in the L304 do not like too high of an input
>>> power.  There is a some kind of effect (that looks thermal but we
>>> were never certain) and results in a slow wandering of phase at the
>>> output of the L304. This occurred way below the rated max input power
>>> from the manufacturer but these devices were designed for analog CATV
>>> transmission where that effect would probably have never been seen.
>>> Rob Long and I discovered and analyzed this back during initial EVLA
>>> outfitting and, based on our tests, set the maximum limit on optical
>>> power at the input jack of those receivers as -18 dBm nominal with an
>>> acceptable range of -17dBm to -20 dBm.  Antenna 26 looked to be about
>>> 3dB hotter than that so it needed to be looked at. Sounds like that
>>> has been done. For some reason,  the alert level on the maximum
>>> optical power level monitor in the L304 is set at -15dBm.  This
>>> really should be changed to -17 dBm.
>>>
>>> This doesn't explain what is occurring at the end of the arms since
>>> those antennas look to be in spec as far as optical power is
>>> concerned.  If this truly is a new effect that hasn't been seen
>>> before, perhaps some additional lab testing is in order to see if
>>> something has degraded with age. Perhaps harmonic distortion of the
>>> signal has gotten worse as components have aged, which would fit with
>>> Barry's suggestion.   We do still have the big spools of fiber, so
>>> tests could be setup in the lab.  Originally, we saw the effect by
>>> comparing the recovered data clock on a deformatter to the original
>>> clock from the L350 using a double balanced mixer along with a scope
>>> / dynamic signal analyzer.   That test was fairly easy when the
>>> deformatters were in the old correlator room during the transition
>>> but would be much harder now with them inside the WIDAR correlator. I
>>> suspect the firmware in them now no longer supports the transition
>>> mode which is what made that possible. In the lab !
>> we!
>>>  could simply measure the L304 output against the original source
>>> fairly easy.
>>>
>>> Cheers,
>>> Jim
>>>
>>> -----Original Message-----
>>> From: evlatests <evlatests-bounces at listmgr.nrao.edu> On Behalf Of
>>> Barry Clark via evlatests
>>> Sent: Saturday, November 2, 2024 12:18 PM
>>> To: evlatests at listmgr.nrao.edu
>>> Subject: Re: [evlatests] phase noise on outer antennas {External}
>>>
>>> The accidental experiment on ea26 suggests to me that the problem
>>> occurs at or near the laser transmitter, and is triggered by a high
>>> laser power.  The ends of the arms are susceptible because those
>>> fibers are driven harder, due to their length.
>>>
>>> When hardware misbehaves in curious ways it is always tempting to
>>> think some sort of thermal effect might be involved.  It might be
>>> worth trying a little extra cooling on the end-of-arm drivers, though
>>> with little expectation of positive results. More likely is the
>>> generation of a LO harmonic which is insufficiently suppressed at the
>>> telescope, and which varies in phase relative to the primary.
>>>
>>> During the EVLA design phase, I remember extensive tests on a big
>>> spool of fiber.  Would these tests have found this effect if it were
>>> present then?
>>>
>>> On 10/31/2024 4:02 PM, Paul Demorest via evlatests wrote:
>>>> After this change the phase jitter on ea26 has decreased to about
>>>> the same level as most other antennas (~1 deg RMS at X-band).
>>>>
>>>> The outer antennas still look the same (~6 deg on ea18 and ea21, ~3
>>>> deg on ea24).
>>>>
>>>> -Paul
>>>>
>>>> ________________________________________
>>>> From: Bruce Mues <bmues at nrao.edu>
>>>> Sent: Thursday, October 31, 2024 3:08 PM
>>>> To: Rob Long; Paul Demorest; Jim Jackson; Ken Sowinski
>>>> Cc: evlatests
>>>> Subject: RE: [evlatests] phase noise on outer antennas
>>>>
>>>> The optical power on ea26 was attenuated this afternoon to be within
>>>> the proper range @ -17.762.
>>>>
>>>> Thanks,
>>>>
>>>> Bruce Mues
>>>> Technical Manager II: Servo-Fiber
>>>> Work Schedule: Tu-Fr, 6:30am-4:30pm
>>>> NRAO-VLA
>>>> New Mexico, USA
>>>> P: 575-835-7417
>>>> E: bmues at nrao.edu
>>>>
>>>> -----Original Message-----
>>>> From: evlatests <evlatests-bounces at listmgr.nrao.edu> On Behalf Of Rob
>>>> Long via evlatests
>>>> Sent: Wednesday, October 30, 2024 5:00 PM
>>>> To: Paul Demorest <pdemores at nrao.edu>; Jim Jackson
>>>> <jjackson at nrao.edu>; Ken Sowinski <ksowinsk at nrao.edu>
>>>> Cc: evlatests <evlatests at nrao.edu>
>>>> Subject: Re: [evlatests] phase noise on outer antennas
>>>>
>>>> Jim and I looked over the antennas and agree that ea26 could be an
>>>> optical level problem (power is too high), but the outer antenna
>>>> levels look normal.
>>>>
>>>> Rob
>>>>
>>>> On 10/30/2024 4:57 PM, Paul Demorest wrote:
>>>>> My guess is probably not given how random these variations look..
>>>>> but we are (for other reasons) planning some tests with these outer
>>>>> antennas plus VLBA that should be interesting.  Online fringe
>>>>> rotation will be disabled and the data will be correlated with
>>>>> difx. If we still see this jitter I think that will mostly rule out
>>>>> any WIDAR-based cause (hopefully Ken agrees with this statement ;)
>>>>> Won't have the results for at least a few weeks though.
>>>>>
>>>>> -Paul
>>>>>
>>>>> ________________________________________
>>>>> From: Rob Long <rlong at nrao.edu>
>>>>> Sent: Wednesday, October 30, 2024 2:53 PM
>>>>> To: Jim Jackson; Paul Demorest; Ken Sowinski
>>>>> Cc: evlatests
>>>>> Subject: Re: [evlatests] phase noise on outer antennas
>>>>>
>>>>> Is there some error being introduced in the fringe rotator for the 3
>>>>> long distance antennas?
>>>>>
>>>>> Rob
>>>>>
>>>>> On 10/30/2024 2:49 PM, Jim Jackson wrote:
>>>>>> Thanks, so the slower jitter on those timescales would likely be
>>>>>> tracked by the L305 oscillator (and synthesizers) and not cleaned
>>>>>> up in the antenna.
>>>>>>
>>>>>> Jim
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Paul Demorest <pdemores at nrao.edu>
>>>>>> Sent: Wednesday, October 30, 2024 2:46 PM
>>>>>> To: Jim Jackson <jjackson at nrao.edu>; Rob Long <rlong at nrao.edu>; Ken
>>>>>> Sowinski <ksowinsk at nrao.edu>
>>>>>> Cc: evlatests <evlatests at nrao.edu>
>>>>>> Subject: Re: [evlatests] phase noise on outer antennas
>>>>>>
>>>>>> These observations were sensitive to jitter on 10ms and slower
>>>>>> timescales.  And when computing RMS I filtered out variations
>>>>>> slower than 1s to remove atmospheric effects. So the observed
>>>>>> jitter is on ~1 to 50 Hz scales (does not mean there is not faster
>>>>>> jitter also, it just gets averaged out in these data).
>>>>>>
>>>>>> -Paul
>>>>>>
>>>>>> ________________________________________
>>>>>> From: Jim Jackson <jjackson at nrao.edu>
>>>>>> Sent: Wednesday, October 30, 2024 2:37 PM
>>>>>> To: Rob Long; Ken Sowinski; Paul Demorest
>>>>>> Cc: evlatests
>>>>>> Subject: RE: [evlatests] phase noise on outer antennas
>>>>>>
>>>>>> It would be interesting to know if the nature of this phase noise
>>>>>> in within or outside the loop bandwidth of the L305 PLL.  With
>>>>>> mention of 10ms dump time this means that jitter is seen within
>>>>>> the 10ms period - ie. it is faster than 10 ms?
>>>>>>
>>>>>> Jim
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: evlatests <evlatests-bounces at listmgr.nrao.edu> On Behalf Of
>>>>>> Rob Long via evlatests
>>>>>> Sent: Wednesday, October 30, 2024 2:28 PM
>>>>>> To: Ken Sowinski <ksowinsk at nrao.edu>; Paul Demorest
>>>>>> <pdemores at nrao.edu>
>>>>>> Cc: evlatests <evlatests at nrao.edu>
>>>>>> Subject: Re: [evlatests] phase noise on outer antennas
>>>>>>
>>>>>> Since the LO is actually cleaned up with a VCXO and PLL at the
>>>>>> antenna, I would think we would see the L305 lose lock if the
>>>>>> optical levels were too low. If there were some phase variation,
>>>>>> we should also see the EFC voltage changing as well (indicating
>>>>>> the reference or VCXO drifting).
>>>>>>
>>>>>> Rob
>>>>>>
>>>>>> On 10/29/2024 8:48 AM, Ken Sowinski via evlatests wrote:
>>>>>>> Since you first brought this up I have wondered whether the strict
>>>>>>> pad dependence implicated a problem with fiber.  Your summary here
>>>>>>> reinforces that as an explanation.  Perhaps a check of LO quality
>>>>>>> at
>>>>>>> W32 (it's easier to get to) is in order?
>>>>>>>
>>>>>>> Ken
>>>>>>>
>>>>>>>
>>>>>>> On Mon, 28 Oct 2024, Paul Demorest via evlatests wrote:
>>>>>>>
>>>>>>>> hi all,
>>>>>>>>
>>>>>>>> Now that we're in A-config, recent testing at high time resolution
>>>>>>>> (10ms) has shown excess short-timescale phase noise affecting the
>>>>>>>> outer three antennas (ea21 at E72, ea24 at W72 and ea18 at N72), as well as
>>>>>>>> ea26 at W32.  A few plots are attached so you can see what I mean.
>>>>>>>> These were done via 10ms-dump-time observations of a bright source
>>>>>>>> at K-band, and show two of the bad antennas (ea18, ea21) plus a
>>>>>>>> good antenna (ea13) for comparison.  I also took data at X and Ka
>>>>>>>> bands, a summary of these results is in the attached text file.
>>>>>>>> The worst ones have short-term phase RMS of ~20 deg at Ka, which
>>>>>>>> is
>>>>>>>>> 10x larger than the good antennas.  This is enough to cause
>>>>>>>> decorrelation (sensitivity loss) at the ~5-10% level and may be at
>>>>>>>> least partially responsible for reduced high-freq performance seen
>>>>>>>> on these antennas in recent stress tests.
>>>>>>>>
>>>>>>>> This noise has the following properties which make me suspect
>>>>>>>> something like LO phase jitter is the cause:
>>>>>>>>     - It looks totally random vs time.
>>>>>>>>     - All 4 IFs for a given antenna show exactly the same noise
>>>>>>>> (see zoom-in plot for example).
>>>>>>>>     - The amplitude of the noise scales in proportion to observing
>>>>>>>> frequency, higher freqs are more affected.
>>>>>>>>
>>>>>>>> It's also notable that all 3 outer antennas looked fine in
>>>>>>>> B-config, and only started showing this noise once they moved to
>>>>>>>> their A-config locations.  I found some 10ms X-band data from last
>>>>>>>> A-config (Aug
>>>>>>>> 2023) and it shows a similar pattern:  the outer pads and W32 are
>>>>>>>> noisier than the rest, even though some different antennas were
>>>>>>>> involved at the time.
>>>>>>>>
>>>>>>>> Since this seems to be pad-related rather than antenna-related, my
>>>>>>>> hand-wavy guess is maybe LO optical power is getting a bit too
>>>>>>>> weak over the very long fibers?  And as for W32 maybe it just has
>>>>>>>> some problem with its fiber connection causing a similar effect.
>>>>>>>> Even if that's not the right explanation, I do think this is worth
>>>>>>>> understanding and (if possible) fixing.  Let me know if you have
>>>>>>>> any questions/suggestions.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Paul
>>>>>>> _______________________________________________
>>>>>>> evlatests mailing list
>>>>>>> evlatests at listmgr.nrao.edu
>>>>>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>>>> _______________________________________________
>>>>>> evlatests mailing list
>>>>>> evlatests at listmgr.nrao.edu
>>>>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>> _______________________________________________
>>>> evlatests mailing list
>>>> evlatests at listmgr.nrao.edu
>>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>>
>>>> _______________________________________________
>>>> evlatests mailing list
>>>> evlatests at listmgr.nrao.edu
>>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>>
>>>
>>> _______________________________________________
>>> evlatests mailing list
>>> evlatests at listmgr.nrao.edu
>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>
>>> _______________________________________________
>>> evlatests mailing list
>>> evlatests at listmgr.nrao.edu
>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>
>>> _______________________________________________
>>> evlatests mailing list
>>> evlatests at listmgr.nrao.edu
>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>
>>
>> _______________________________________________
>> evlatests mailing list
>> evlatests at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>
> _______________________________________________
> evlatests mailing list
> evlatests at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/evlatests

More information about the evlatests mailing list