[evlatests] phase noise on outer antennas {External}

Paul Demorest pdemores at nrao.edu
Wed Nov 13 19:25:36 EST 2024 

Thanks, I got some more 10ms data today to verify, will report back tomorrow.

In the meantime, a week or so ago, I recorded raw data from four antennas (ea11, 13, 18, 22) to get a look at higher time resolution.  Two inner and two outer antennas, and the only "bad" one in this set is ea18.  I processed this at 0.5 ms to be able to see fluctuation frequencies up to 1 kHz.  Attached plot shows a power spectrum of the phase noise on all 6 baselines, with 0.1 Hz resolution.  A few notable things:

1. The ea18 baselines all show the excess phase noise, as seen before.  The amplitude starts dropping around 50 Hz and gets down to a comparable level to the good baselines around 200 Hz.  I think this is all expected given a PLL bandwidth of 50 Hz previously noted in this thread.

2. In addition to the broad-band noise there are some notable spikes in the ea18 data at 19.2, 26.2, 29.7, 39.3, and 59.5 Hz.  Aside from that last one being near 60 Hz power-line freq, none of these numbers are especially meaningful to me.  Do they ring a bell for any of you?

3. The "good" baselines show a few spikes as well, at 4.0, 10.0 and 20.0 Hz.   Though at a much lower level so not really a problem more of a curiosity.  I believe 4 Hz is a known correlator artifact related to the length of a memory buffer (Ken can probably confirm).  And 10 Hz and harmonics are common throughout the VLA for multiple reasons, and have been noted many times before.

Cheers,
Paul

________________________________________
From: evlatests <evlatests-bounces at listmgr.nrao.edu> on behalf of Leon Abeyta via evlatests <evlatests at listmgr.nrao.edu>
Sent: Wednesday, November 13, 2024 2:13 PM
To: evlatests at listmgr.nrao.edu
Cc: Jeffrey Yarnell; Michael LeBlanc
Subject: Re: [evlatests] phase noise on outer antennas {External}

The patch panel for the 3 farthest antennas were checked today
(11/13/24) and they were all without attenuators. Attenuators labelled 2
dB were added to the 3 antennas in the hope that this will improve the
phase noise.

Leon Abeyta


On 11/7/2024 9:12 AM, Ken Sowinski via evlatests wrote:
> On Tue, 5 Nov 2024, Jim Jackson via evlatests wrote:
>> Thanks Paul.
>>
>> Usually this has been accomplished by adding an attenuator in the
>> optical path.  The easiest place to do that is in the patch panel
>> rack in the Control Building right after the L353 transmitters on the
>> fiber going out to the antenna.
>
> This suggests that it is happening in the long fiber run or the detetor
> at the antenna; not the optical transmitter.  Would you learn anything
> by comparing the returned signal on the RT phase fiber with the optical
> output of the L353?
>
> Ken
>
> ]
>> The effect Rob and I analyzed years ago was clearly related to the
>> optical power level at the input jack of the optical receiver in the
>> L304.  I'll check with Leon to confirm exactly what they did.  I
>> suspect they added optical attenuation at the patch panel in the
>> building to get the measured optical power in the L304 within the -17
>> to -20 dBm spec, then reduced the RF attenuation between the L304 and
>> L305 in the antenna to get the RF power into the L305 at -26.5 dBm
>> where it likes to be.
>>
>> Cheers,
>> Jim
>>
>>
>>
>> -----Original Message-----
>> From: Paul Demorest <pdemores at nrao.edu>
>> Sent: Tuesday, November 5, 2024 4:27 PM
>> To: Barry Clark <bclark at nrao.edu>; evlatests <evlatests at nrao.edu>;
>> Jim Jackson <jjackson at nrao.edu>
>> Subject: Re: [evlatests] phase noise on outer antennas {External}
>>
>> Thanks Jim.  A couple follow-ups:
>>
>> I did the test suggested by Ken of comparing a far-North source
>> (J1800+7828) with one near the equator (3C273 aka J1229+0203). The
>> fringe rate due to earth rotation decreases by about a factor of 5
>> (proportional to cos(dec)) between these two.  No difference was
>> observed in the amplitude or appearance of the outer antenna phase
>> jitter; everything looks about the same.  I think this continues to
>> suggest that this is not a correlator-based effect.
>>
>> I think Barry's suggestion that this effect might occur at the
>> transmit side of the LO fiber makes a lot of sense given what we've
>> observed so far.  When the ea26 power level was reduced was that a
>> change at the transmitter, or was an attenuator installed at the
>> antenna?
>>
>> Also the ea18 optical power is very slightly outside your recommended
>> range: it's -16.2 dBm, which is now the highest value on the array
>> after the ea26 change.  However the other two arm-end antennas are
>> much lower (ea21=-18.3 and ea24=-19.4).
>>
>> Cheers,
>> Paul
>>
>> ________________________________________
>> From: evlatests <evlatests-bounces at listmgr.nrao.edu> on behalf of Jim
>> Jackson via evlatests <evlatests at listmgr.nrao.edu>
>> Sent: Sunday, November 3, 2024 12:03 PM
>> To: Barry Clark; evlatests
>> Subject: Re: [evlatests] phase noise on outer antennas {External}
>>
>> I spoke to Leon about this last week.  We know that the Fiber Span
>> optical receivers in the L304 do not like too high of an input
>> power.  There is a some kind of effect (that looks thermal but we
>> were never certain) and results in a slow wandering of phase at the
>> output of the L304. This occurred way below the rated max input power
>> from the manufacturer but these devices were designed for analog CATV
>> transmission where that effect would probably have never been seen.
>> Rob Long and I discovered and analyzed this back during initial EVLA
>> outfitting and, based on our tests, set the maximum limit on optical
>> power at the input jack of those receivers as -18 dBm nominal with an
>> acceptable range of -17dBm to -20 dBm.  Antenna 26 looked to be about
>> 3dB hotter than that so it needed to be looked at. Sounds like that
>> has been done. For some reason,  the alert level on the maximum
>> optical power level monitor in the L304 is set at -15dBm.  This
>> really should be changed to -17 dBm.
>>
>> This doesn't explain what is occurring at the end of the arms since
>> those antennas look to be in spec as far as optical power is
>> concerned.  If this truly is a new effect that hasn't been seen
>> before, perhaps some additional lab testing is in order to see if
>> something has degraded with age. Perhaps harmonic distortion of the
>> signal has gotten worse as components have aged, which would fit with
>> Barry's suggestion.   We do still have the big spools of fiber, so
>> tests could be setup in the lab.  Originally, we saw the effect by
>> comparing the recovered data clock on a deformatter to the original
>> clock from the L350 using a double balanced mixer along with a scope
>> / dynamic signal analyzer.   That test was fairly easy when the
>> deformatters were in the old correlator room during the transition
>> but would be much harder now with them inside the WIDAR correlator. I
>> suspect the firmware in them now no longer supports the transition
>> mode which is what made that possible. In the lab !
> we!
>>  could simply measure the L304 output against the original source
>> fairly easy.
>>
>> Cheers,
>> Jim
>>
>> -----Original Message-----
>> From: evlatests <evlatests-bounces at listmgr.nrao.edu> On Behalf Of
>> Barry Clark via evlatests
>> Sent: Saturday, November 2, 2024 12:18 PM
>> To: evlatests at listmgr.nrao.edu
>> Subject: Re: [evlatests] phase noise on outer antennas {External}
>>
>> The accidental experiment on ea26 suggests to me that the problem
>> occurs at or near the laser transmitter, and is triggered by a high
>> laser power.  The ends of the arms are susceptible because those
>> fibers are driven harder, due to their length.
>>
>> When hardware misbehaves in curious ways it is always tempting to
>> think some sort of thermal effect might be involved.  It might be
>> worth trying a little extra cooling on the end-of-arm drivers, though
>> with little expectation of positive results. More likely is the
>> generation of a LO harmonic which is insufficiently suppressed at the
>> telescope, and which varies in phase relative to the primary.
>>
>> During the EVLA design phase, I remember extensive tests on a big
>> spool of fiber.  Would these tests have found this effect if it were
>> present then?
>>
>> On 10/31/2024 4:02 PM, Paul Demorest via evlatests wrote:
>>> After this change the phase jitter on ea26 has decreased to about
>>> the same level as most other antennas (~1 deg RMS at X-band).
>>>
>>> The outer antennas still look the same (~6 deg on ea18 and ea21, ~3
>>> deg on ea24).
>>>
>>> -Paul
>>>
>>> ________________________________________
>>> From: Bruce Mues <bmues at nrao.edu>
>>> Sent: Thursday, October 31, 2024 3:08 PM
>>> To: Rob Long; Paul Demorest; Jim Jackson; Ken Sowinski
>>> Cc: evlatests
>>> Subject: RE: [evlatests] phase noise on outer antennas
>>>
>>> The optical power on ea26 was attenuated this afternoon to be within
>>> the proper range @ -17.762.
>>>
>>> Thanks,
>>>
>>> Bruce Mues
>>> Technical Manager II: Servo-Fiber
>>> Work Schedule: Tu-Fr, 6:30am-4:30pm
>>> NRAO-VLA
>>> New Mexico, USA
>>> P: 575-835-7417
>>> E: bmues at nrao.edu
>>>
>>> -----Original Message-----
>>> From: evlatests <evlatests-bounces at listmgr.nrao.edu> On Behalf Of Rob
>>> Long via evlatests
>>> Sent: Wednesday, October 30, 2024 5:00 PM
>>> To: Paul Demorest <pdemores at nrao.edu>; Jim Jackson
>>> <jjackson at nrao.edu>; Ken Sowinski <ksowinsk at nrao.edu>
>>> Cc: evlatests <evlatests at nrao.edu>
>>> Subject: Re: [evlatests] phase noise on outer antennas
>>>
>>> Jim and I looked over the antennas and agree that ea26 could be an
>>> optical level problem (power is too high), but the outer antenna
>>> levels look normal.
>>>
>>> Rob
>>>
>>> On 10/30/2024 4:57 PM, Paul Demorest wrote:
>>>> My guess is probably not given how random these variations look..
>>>> but we are (for other reasons) planning some tests with these outer
>>>> antennas plus VLBA that should be interesting.  Online fringe
>>>> rotation will be disabled and the data will be correlated with
>>>> difx. If we still see this jitter I think that will mostly rule out
>>>> any WIDAR-based cause (hopefully Ken agrees with this statement ;)
>>>> Won't have the results for at least a few weeks though.
>>>>
>>>> -Paul
>>>>
>>>> ________________________________________
>>>> From: Rob Long <rlong at nrao.edu>
>>>> Sent: Wednesday, October 30, 2024 2:53 PM
>>>> To: Jim Jackson; Paul Demorest; Ken Sowinski
>>>> Cc: evlatests
>>>> Subject: Re: [evlatests] phase noise on outer antennas
>>>>
>>>> Is there some error being introduced in the fringe rotator for the 3
>>>> long distance antennas?
>>>>
>>>> Rob
>>>>
>>>> On 10/30/2024 2:49 PM, Jim Jackson wrote:
>>>>> Thanks, so the slower jitter on those timescales would likely be
>>>>> tracked by the L305 oscillator (and synthesizers) and not cleaned
>>>>> up in the antenna.
>>>>>
>>>>> Jim
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: Paul Demorest <pdemores at nrao.edu>
>>>>> Sent: Wednesday, October 30, 2024 2:46 PM
>>>>> To: Jim Jackson <jjackson at nrao.edu>; Rob Long <rlong at nrao.edu>; Ken
>>>>> Sowinski <ksowinsk at nrao.edu>
>>>>> Cc: evlatests <evlatests at nrao.edu>
>>>>> Subject: Re: [evlatests] phase noise on outer antennas
>>>>>
>>>>> These observations were sensitive to jitter on 10ms and slower
>>>>> timescales.  And when computing RMS I filtered out variations
>>>>> slower than 1s to remove atmospheric effects. So the observed
>>>>> jitter is on ~1 to 50 Hz scales (does not mean there is not faster
>>>>> jitter also, it just gets averaged out in these data).
>>>>>
>>>>> -Paul
>>>>>
>>>>> ________________________________________
>>>>> From: Jim Jackson <jjackson at nrao.edu>
>>>>> Sent: Wednesday, October 30, 2024 2:37 PM
>>>>> To: Rob Long; Ken Sowinski; Paul Demorest
>>>>> Cc: evlatests
>>>>> Subject: RE: [evlatests] phase noise on outer antennas
>>>>>
>>>>> It would be interesting to know if the nature of this phase noise
>>>>> in within or outside the loop bandwidth of the L305 PLL.  With
>>>>> mention of 10ms dump time this means that jitter is seen within
>>>>> the 10ms period - ie. it is faster than 10 ms?
>>>>>
>>>>> Jim
>>>>>
>>>>>
>>>>>
>>>>> -----Original Message-----
>>>>> From: evlatests <evlatests-bounces at listmgr.nrao.edu> On Behalf Of
>>>>> Rob Long via evlatests
>>>>> Sent: Wednesday, October 30, 2024 2:28 PM
>>>>> To: Ken Sowinski <ksowinsk at nrao.edu>; Paul Demorest
>>>>> <pdemores at nrao.edu>
>>>>> Cc: evlatests <evlatests at nrao.edu>
>>>>> Subject: Re: [evlatests] phase noise on outer antennas
>>>>>
>>>>> Since the LO is actually cleaned up with a VCXO and PLL at the
>>>>> antenna, I would think we would see the L305 lose lock if the
>>>>> optical levels were too low. If there were some phase variation,
>>>>> we should also see the EFC voltage changing as well (indicating
>>>>> the reference or VCXO drifting).
>>>>>
>>>>> Rob
>>>>>
>>>>> On 10/29/2024 8:48 AM, Ken Sowinski via evlatests wrote:
>>>>>> Since you first brought this up I have wondered whether the strict
>>>>>> pad dependence implicated a problem with fiber.  Your summary here
>>>>>> reinforces that as an explanation.  Perhaps a check of LO quality
>>>>>> at
>>>>>> W32 (it's easier to get to) is in order?
>>>>>>
>>>>>> Ken
>>>>>>
>>>>>>
>>>>>> On Mon, 28 Oct 2024, Paul Demorest via evlatests wrote:
>>>>>>
>>>>>>> hi all,
>>>>>>>
>>>>>>> Now that we're in A-config, recent testing at high time resolution
>>>>>>> (10ms) has shown excess short-timescale phase noise affecting the
>>>>>>> outer three antennas (ea21 at E72, ea24 at W72 and ea18 at N72), as well as
>>>>>>> ea26 at W32.  A few plots are attached so you can see what I mean.
>>>>>>> These were done via 10ms-dump-time observations of a bright source
>>>>>>> at K-band, and show two of the bad antennas (ea18, ea21) plus a
>>>>>>> good antenna (ea13) for comparison.  I also took data at X and Ka
>>>>>>> bands, a summary of these results is in the attached text file.
>>>>>>> The worst ones have short-term phase RMS of ~20 deg at Ka, which
>>>>>>> is
>>>>>>>> 10x larger than the good antennas.  This is enough to cause
>>>>>>> decorrelation (sensitivity loss) at the ~5-10% level and may be at
>>>>>>> least partially responsible for reduced high-freq performance seen
>>>>>>> on these antennas in recent stress tests.
>>>>>>>
>>>>>>> This noise has the following properties which make me suspect
>>>>>>> something like LO phase jitter is the cause:
>>>>>>>     - It looks totally random vs time.
>>>>>>>     - All 4 IFs for a given antenna show exactly the same noise
>>>>>>> (see zoom-in plot for example).
>>>>>>>     - The amplitude of the noise scales in proportion to observing
>>>>>>> frequency, higher freqs are more affected.
>>>>>>>
>>>>>>> It's also notable that all 3 outer antennas looked fine in
>>>>>>> B-config, and only started showing this noise once they moved to
>>>>>>> their A-config locations.  I found some 10ms X-band data from last
>>>>>>> A-config (Aug
>>>>>>> 2023) and it shows a similar pattern:  the outer pads and W32 are
>>>>>>> noisier than the rest, even though some different antennas were
>>>>>>> involved at the time.
>>>>>>>
>>>>>>> Since this seems to be pad-related rather than antenna-related, my
>>>>>>> hand-wavy guess is maybe LO optical power is getting a bit too
>>>>>>> weak over the very long fibers?  And as for W32 maybe it just has
>>>>>>> some problem with its fiber connection causing a similar effect.
>>>>>>> Even if that's not the right explanation, I do think this is worth
>>>>>>> understanding and (if possible) fixing.  Let me know if you have
>>>>>>> any questions/suggestions.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Paul
>>>>>> _______________________________________________
>>>>>> evlatests mailing list
>>>>>> evlatests at listmgr.nrao.edu
>>>>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>>> _______________________________________________
>>>>> evlatests mailing list
>>>>> evlatests at listmgr.nrao.edu
>>>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>> _______________________________________________
>>> evlatests mailing list
>>> evlatests at listmgr.nrao.edu
>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>
>>> _______________________________________________
>>> evlatests mailing list
>>> evlatests at listmgr.nrao.edu
>>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>>
>>
>> _______________________________________________
>> evlatests mailing list
>> evlatests at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>
>> _______________________________________________
>> evlatests mailing list
>> evlatests at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>
>> _______________________________________________
>> evlatests mailing list
>> evlatests at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/evlatests
>>
>
> _______________________________________________
> evlatests mailing list
> evlatests at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/evlatests

_______________________________________________
evlatests mailing list
evlatests at listmgr.nrao.edu
https://listmgr.nrao.edu/mailman/listinfo/evlatests
-------------- next part --------------
A non-text attachment was scrubbed...
Name: phase_noise_psd.png
Type: image/png
Size: 154565 bytes
Desc: phase_noise_psd.png
URL: <http://listmgr.nrao.edu/pipermail/evlatests/attachments/20241114/ce70b9f4/attachment-0001.png>

More information about the evlatests mailing list