[evla-sw-discuss] System design decisions needed for Widar

[Barry Clark]

There are several issues to be decided about what we want the system
to look like in the long and intermediate term so we know what
general direction to take in the short run.

1.  Flags from the telescopes.
The long term route is clear - they should be collected by MCAF
and inserted into the SDM.  There are problems with doing this
in the short term because of a breakdown in the colaboration
with ALMA.
Alternatives in the short term are
A.  Have MCAF put the flags in a temporary flag table in the SDM
     which may or may not morph into the final flag table when ALMA
     is interested in talking about it.  Table would be handled by
     the CASA filler in 'transparent mode', and a special CASA program
     written to run after the filler to compute the flags and put them
     in the measurement set.
B.  Writing a separate flag handling program to bundle them up, label
     them appropriately, and ship them off to the CBE to apply as
     baseline based flags.  (Suffers the disadvantage of creating a new
     data route that will be abandoned in the long run.)

2.  Setting the filter requantizer gains
Alternatives are
A.  Continuous ALC.  I would strongly recommend that the ALC loop be
     based on an integration boxcar synchronous and conterminous with
     integrations.  Anything else is just too hard to keep track of.
     Then the station board product is the requantizer gain, which must
     be recorded every integration to be able to tie things together.
B.  ALC for each scan.  (My favorite.)  This is the way the attenuators
     in the T304s are set.  In this case, the output of the station
     board is the requantizer gain once per scan (used for bandpass
     stitching), and the requantizer power.  This option raises the
     question of by what route the station board is notified that a
     new scan is occurring.
C.  Set gains to a standard value.  Then the output of the station board
     is the requantizer power, used for converting to correlation
     coefficients.  This runs into trouble when subbands have very different
     power levels.
In any of these cases we probably need a program to collect the outputs
from the station boards, package them up and send somewhere useful.  The
'somewhere useful' probably includes MCAF to include in the SDM.  Even if
we decide that these data products are to be used in the CBE, rather than
CASA, they probably need to be included in the SDM to provide a record of
what was done.

At a guess, we'll end up doing all three options above.  C is what we
have now and is therefore the default, B produces the easiest data to
process, so we are likely to want to do it for now, and A may be insisted
upon by a solar observer if we ever have one.

3.  System/Noise source application.
I believe it is generally agreed that the ratio of system to cal be
included in the SDM for application during post processing.  This
probably implies a listener within the correlator system to collect
these, package them, and export to MCAF.  Again, I strongly urge that
these be collected and recorded once per integration.

4.  Are correlation coefficients useful?
 From a sufficiently high level abstract viewpoint, the answer is no.
Calibration is done by dividing each subband by the subband gain, and
then multiplying that whole combined spectrum by a number that converts
it to Janskys.  This number is derived by looking at the "best"
subband (least interference, best known cal) and dividing by the
measured cal power (that is, cal on minus cal off), and multiplying
by the Tcal for that subband.  This converts the measure correlations
into kelvin.  One then multiplies by the antenna effective area which
converts them into Janskys.  (but see footnote)  The power level out
of the requantizer nowhere enters in this description of calibration.
On the other hand, for various intermediate operations, particularly
in telcal, having correlation coefficients, derived by dividing the
measured correlations by the power out of the requantizer, really
simplifies things by not having to worry about changes in power level
across the time of its calculations.
If we decide correlation coefficients are useful, it raises the question
of when they are calculated - in the CBE or later in the stream.  (If
this division is done in the CBE, we will have to have the power of
undoing it later, since, as noted above, correlation coefficients are
not used in the calibration process.)

5.  Delay residuals.
The see-sawing of phase across the band due to the delay cogging is
alleged to be a problem, and it probably is at some very high dynamic
range, and becomes a nuisance in more ordinary observations.  We
should plan to correct it, although it is not clear to me that there
is a very high current priority on correcting it.  I can see two
ways of making the correction.
A.  Make available the delay polynomials used at observe time, and
     replicate the calculations done on the station board to estimate
     the mean phase slope residuals during the integration to calculate
     the correction.  The delay polynomials may be captured from the
     Executor multicast or relayed from the station board, or from the
     polynomial distributor.
B.  The station board may be taxed with the job of producing the average
     and RMS delay error over the integration, and this can be provided as
     one of the station board products.  A rough calculation indicates that
     correcting the delay by the mean error and the amplitude by the RMS
     error should be sufficient even for quite high dynamic ranges.
In either case, the correction may be done in the CBE, or later, in CASA,
determining whether the data stream is sent to the CBE or to the SDM.

6.  Tcal.
We probably, in the long run, need cal spectra.  If we decide we do,
probably the most reasonable place to store them is in a new table
in the evlaparm database.  We might need a receiver serial number
in the main parameters table to simplify the situation if a receiver
is removed from one antenna and installed in another, though it would
not be too complicated to keep track of this by hand.  The question
that arises is who reads the table - Executor or MCAF.  (One might
also palm this off on Scheduler.)  If the Executor, when should it
be read - every scan?  However, as noted above, we really need a Tcal
only for one subband for each sampler.  If we get properly organized,
we can probably live with the existing system for a long time.

7.  Scans and subscans.
The CBE must know about scans and subscans, obviously.  It is currently
told about them directly by the Executor.  Is this intended to be the
final route, or is this a temporary expedient?  If permanent, we need
to know the information to be added to let the CBE sort out the multiple
subarray case.  Also, as mentioned above, it is perhaps expedient to be
able to tell subband filters "This is a new scan - set your output power
level to something sensible." This clearly has to go through the VCI
interface - any problems with that?  We need to decide the message content.
In particular, we would rather like to send a single message, and have
the ConfigurationMapper keep track of what station boards to forward
it to.
(also see footnote 2)

8.  How does correlator configuration information get from the OPT to
the correlator?
The plan is for the scheduler to produce an Executor script and
correlator configuration documents.  I can see two ways for the
documents to get to the correlator.
A.  The scheduler may write files containing the documents and insert
     the file names in the script.  The Executor would then send the
     document to the VCI as needed, either through a socket or by passing
     a file name, as the current human interface to the ConfigurationMapper
     does.
B.  The scheduler creates a queue of correlator configuration times,
     and delayed processes that wake up and send configurations to the
     VCI at the appropriate times.
It seems to me that A is the best way to do this.  Among several other
reasons, it trivially permits human edited scripts, which would be a
bear via route B.

9.  How does configuration information get from the OPT to the CBE?
This is the ConfigurationMapper, whether one chooses to call it that
or not.  If you choose to write it as a separate entity, you will
have to worry a lot about communication protocols between the two
pieces.  From an ethereal plane, one might simply recommend
combining ConfigurationMapper and the CBE master.  Even on a less
ethereal plane, it would seem to me better to make it as much as
possible one piece, so that things currently only in the CBE
master get put in ConfigurationMapper, like a list of available nics
and ports would be kept in the ConfigurationMapper.  Then the
ConfigurationMapper can organize sending the addresses to the
baseline boards.  It will also have to pass along to the CBE slave
processes what it has done, and what processing they need to do.
(Note:  here I am interfering in an area I know nothing about, so
the content may be inane, but something needs to be decided and
written down.)

Footnote 1.  Instead of simply the switched power, sending along
(Pon - Poff)/(Pon + Poff)*(Requantizer_Power)/Requantizer_Gain
cancels a few mildly annoying fiddly corrections.

Footnote 2.  Despite the fact that scan_number, subscan_number are
hermetic within the BDF/SDM system and do not leak out, Rich and
Martin have been urging to have the Executor provide them, on the
grounds that associating data via an integer is nicer than via the
floating point time.  It is reasonable for the Executor to do this,
but one would prefer not to unless the current direct communication
scheme to the CBE is the long term one.