[widar-wg] Finished with B108-b-XX boards; test results; H/W faults found :-(

[Brent Carlson]

I'm finished testing with the bottom crate of B108.

Some things that were found:

1. There were some anomalies in tests 11, 12, 13, and 14, and it turns 
out in 7w-10w as well.  These didn't have anything to do with phasing 
testing chips, but with the rest of the array, which may have resulted 
in some strangeness.  These files are now fixed and installed in mccc 
ok, and test ok such that I believe the test xml files are ok.

2. There was a bug in the xml config file/the gui writing to the file 
which put the phasing logic in the RXPs in agc mode, when for direct bit 
exact comparison, it must be a set gain and no agc mode.  Bruce fixed 
this.  There was also subsequently found a race condition in setting the 
gain and agc off, which Bruce has also fixed.

3. The remaining issue seems to be the corr chip "bit 28" problem, 
seemingly caused by the CRM issuing a "user re-phase" command, causing 
some corr chips to hang, not produce data, and therefore fail the 
tests.  Bruce has seen this smoking gun, and is looking into it.  Seems 
to only be associated with corr chips in autocorr mode, which these 
tests use.

4. Board b108-b-6 seems to have a hard correlator chip failure, X0Y1, 
and should be replaced at the next opportunity.

5. Board b108-b-4 seems to have a transmit connection problem with the 
CBE switch, probably due to a cabling fault, but could be anywhere 
including traces on the PCB, or the SFP module itself.  The outgoing 
packets are all addressed fine, but the CRM/mccc packet reader thing 
fails to detect them, whereas it is fine for other boards.  The "SFP1" 
LED in the GigE GUI is green, but this just means the receiver is 
getting good signal and codes from the switch.  Can someone look at the 
switch port for b108-b-4?  If it is ok, then it is a b108-b-4 PCB 
problem (because the SFP module can still put out good codes even if the 
signal into from the GigE FPGA is dead), if not it is a cabling or SFP 
module problem.

Once Bruce gets a solution to 3. in place, we can retest, and then I 
believe things should work ok.

--Brent

-- 
Brent R. Carlson
Brent.Carlson at nrc-cnrc.gc.ca
Tel: 250-497-2346                  |  Fax: (250) 497-2355
Design Engineer                    |  Ingenieur Concepteur
National Research Council Canada   |  Conseil national de recherches Canada
Dominion Radio Astrophysical Obs.  |  Observatoire federal de radioastrophysique
P.O. Box 248, 717 White Lake Rd    |  C.P. 248, 717 Rue White Lake
Penticton, BC, Canada V2A 6K3      |  Penticton, (C.-B.), Canada V2A 6K3
Government of Canada               |  Gouvernement du Canada

"When and where humans are involved, mistakes inevitably happen"