[evla-sw-discuss] EVLA M&C Transition SW Development Plan
Ken Sowinski
ksowinsk at aoc.nrao.edu
Mon Mar 28 15:40:51 EST 2005
----------
X-Sun-Data-Type: text
X-Sun-Data-Description: text
X-Sun-Data-Name: text
X-Sun-Charset: us-ascii
X-Sun-Content-Lines: 64
While this document is a nice list of things that need to get done,
there is no sense of an overview of how all the pieces fit together.
I offer my view of a big picture. Rather than rewrite the document
I will just provide a list of additional assumptions which are either
implicit in what Bill wrote or are merely an indication of how I
think we should proceed. I leave the job of drawing inferences as
an exercise.
I will sometimes use different terms than were employed in the
document and will try to define them when I do so. For now, I declare
that we have a 'current' system, the Modcomps, an 'interim' system, which
is what we operate now for EVLA antennas and a 'final' system which is
not yet designed. 'Final' is used in a limited sense here to mean the
system we intend to deploy as the EVLA monitor and control system, not
that it will have full functionality when first deployed.
Additional assumptions or constraints, annotated.
1. The life of the interim system will be co-terminous with the
life of the Modcomps. A consequence is that the interim system
need not try hard to run VLA antennas and the old correlator. It
may choose to do so for the value of the lessons learned in the
attempt.
2. The final system need not be encumbered with the need to interact
with the Modcomps as the Modcomps will disappear with the interim
system.
3. The final system will produce archive records in the same form as
we know now as long as it provides data from the old correlator. This
gives rise to a tension between the desire to build a stateless system
and the desire to create an output record which is full of state. I
suggest that the tension be relieved in the script, not the executor.
I hinted at this when I described the archive format last february, and
this question needs to be looked at more carefully.
4. When the prototype new correlator is in use, the final system may
produce old archive records (with a structural limit of 32768 channels)
or other new forms as required by Brent, or both. The former allows
for easily passing the data to AIPS along with all the anciillary
information it is used to.
5. DCAF and Telcal in the final system, while still using the old
correlator, will be whatever it takes to provide the functionality we
are accustomed to with the current system. These will likely bear only
slight relation to the correspondingly named systems for use with the
new correlator. The goal should be to spend as little effort as possible
making the old correlator work so that most effort can be devoted to
design and construction of the real final system.
6. Embryonic versions of the real DCAF and Telcal will be required
at the time of the limited WIDAR installation at the VLA, 2008Q1.
That is only three years from now!
At the moment, I think we have some idea of how to build an executor
and control antennas and, maybe, correlators, but little thought has
been given to the systems called DCAF and Telcal. Most of what I have
to say here is to describe what we will need from these systems at the
stages along the way. I have done that in the email that Bill referred,
which had only limited distribution, so I include it in this email as
an attachment.
----------
X-Sun-Data-Type: default
X-Sun-Data-Description: default
X-Sun-Data-Name: telcal
X-Sun-Charset: us-ascii
X-Sun-Content-Lines: 244
A Telcal Stream of Consciousness
Feb 14, 2005
While there is some notion of what Telcal means in the Widar/e2e
era, we have no clear idea of what we mean by Telcal in the short term.
This note is an attempt to clarify what we will do in the pre-Widar era.
While we can describe what we are doing in the Modcomp era, we need a
precise statement of how phasing and referenced pointing will work when
the Modcomps are gone. This note attempts that discussion and is
limited to transition time only. When used without qualification I will
always mean "while using the old correlator" when I say transition.
It is suggested that we never pass telcal information from the Modcmops
to mchost.
Assumptions
Observing scripts for the new system will be generated by obs2py from
(j)observe files. This will be true as long as the old correlator is
in use. As an aside, observing scripts for testing the protoype Widar
will have to be hand crafted or created by some program that does not
yet exist. I can imagine writing an obs2widar, but don't really want to.
We will retain the current archive format for VLA data with the old
correlator as long as the old correlator is in use.
The primary data flow from the VLA online system is through the
network to igloo, a linux system which buffers the data on its way to
the archive. This will remain the case, even after the Modcomps are
retired, as long as we are using the old correlator. Despite this
we do not require igloo to be a part of the current realtime system
because if igloo fails the archive can still be constructed post facto
from the data recorded to tape.
Neglecting details, the transition system after the demise of the
Modcomps will consist of three systems that, though they may be more
complicated, can be represented by mchost, the new system controller
(prospero), and igloo. Depending on system topology any of these are
candidates for the future home of data capture (old DUMP) and Telcal.
If telcal is moved to igloo, igloo will perforce be a required part
of the realtime ssystem.
The transition era is defined to be that period between pure VLA (now!)
and the retirement of the old corrlelator and can be divided into
several stages.
Pre-eVLA
By Telcal we mean ANTSOL and PTGSOL in the Modcomps and the suite
of programs that provide some view of what they are doing. While
observing we provide a crude realtime indication of data quality while
observing calibrators, phase solutions that can be used to phase
the array, and pointing offset measurements that can be used to
correct for pointing errors locally in space and time. Both these
programs are parasitic on the archive data stream. ANTSOL is written
to be so in a strict sense; PTGSOL can be, but accesses other information
in an effort to improve its robustness.
mchost/Modcomps cooperatively
Everything above is still true but if we want to feed information to
mchost for realtime use, Boss will have to learn to provide it.
mchost/Modcomps with new system controller
Everything above is still true. However now we can choose whether to
create the archive record in the Modcomps or in the system controller.
If we have the entire archive record available in the system controller,
the option of implementing ANTSOL and PTGSOL in the system controller
exists.
No Modcomps!
I am assuming that the Telcal functionality we want will be the same as
before with some unclear mumbling about delays and focus. Telcal will
perforce have to exist in one of the three places listed earlier, and
it must of necessity reside in a place where the full archive record
is available: igloo or prospero.
Practicalities
We will have to decide on the details of what is communicated and what
is done with the information to make phasing and referenced pointing
work in the transitional future. To clarify the discussion, I want to
describe what happens now. If for no other reason, to come to an
agreement on what the Modcomps ought to supply to mchost.
both ANTSOL and PTGSOL are simple programs that produce antenna based
results and leave them in a place accessible to anybody who wants to
see it. ANTSOL produces an array of complex gains for each IF/antenna
along with an array to identify which subarray each antenna is associated
with. This information is updated in its entirety every integration while
a calibrator is observed. PTGSOL calculates pointing offsets for each
antenna for each trial (set of five measurements) and adds it to a running
sum of offsets which was cleared at the beginning of the scan. Also
recorded is the number of successful trials, the band, the time and
position on the sky. The information is complete when a pointing scan
ends and is not updated again until the start of a new pointing scan when
it is all cleared.
When specified by observing mode phasing is accomplished by fetching
the recorded phase each integration, filtering it, and leaving the result
to be used by the code that generates the parameters to be sent to the
fringe rotators. For the IFs which are not directly connected to a
fringe rotator, a correction is calculated and passed to the correlator
backend (AP) to correct the phases after correlation.
Referenced pointing is applied by the new source program when specified
in the observe file. It calculates an average offset for each antenna
and adds them, as collimations, to the pointing model. The timing is
arranged so that the scan immediately following a pointing scan will have
the pointing corrected properly.
In both cases, the usage of the data is distinct from its generation.
That leads to obvious questions that can be asked if the Modcmops are
to supply the Telcal data to mchost.
1. Do we always pass along all ANTSOL solutions to let mchost make of
them what it will, or only pass phase corrections when the Modcomp creates
them in response to being in a phasing mode?
2. Do we always pass along all pointing results and let mchost make sense
of it, or only pass along the collimation corrections when they are
generated at the beginning of a new scan that requires referenced pointing?
Schedules
The Transition Observing System Plan defined several phases along
the way to retiring the Modcomps with dates derived from the realities
of hardware availability and the desire to retire the Modcomps as soon
as possible. The short descriptions given for each phase give the major
requirements, but hide a multitude of sins.
Phase 1 - Jan 2005: Usable eVLA antenna available. To use it for
astronomy the goals for Phase 1 must be attained.
Requires Modcomps to supply Telcal functions.
Phase 2 - Jun 2005: New system controller commissioned. Migrate
Telcal functions from the Modcmops.
Phase 3 - Dec 2005: Use CMP to control VLA hardware. Control the
old correlator from mchost.
We are now half way through February and a realistic expectation is that
we will not have an eVLA antennna suitable for astronomical purposes
until July 2005. However it is not just the hardware which is behind
schedule, we are not yet able to send Telcal results from Boss to mchost.
To avoid wasted effort and still allow us to maintain the previous
schedule we should abandon that goal of phase 1 and immediately work
toward migrating Telcal from the Modcomps. If we place it in igloo
which sees all the data anyway, then we will be ready for the end of
phase 3. While the Modcomps exist, Telcal will exist in both Boss and
igloo so that both Boss and mchost will independently have access to
the results.
Topology
The question we have to address is where do data capture (the old DUMP)
and Telcal reside during the successive stages of transition. The
following diagrams are examples of how the system can be organized
at various stages along the way. The first and last unavoidable, but
we need only one of the two intermediate sketches. Thin lines represent
the flow of information necessary to create an archive record or the
results of Telcal feed back to the observing process. Thick lines
represent the flow of visibility data and after DC the archive record.
Parenthetical initials indicate where data capure and Telcal reside.
Modcompcentric:
/\
||
||
_________ ________
| | | |
| igloo | |mchost|
|_______| /--->|______|
/\ /
|| /
|| /
|| /
_________/ __________
(DC) | | | |
(TC) | Boss |<========|Prospero|
|_______| |________|
SystemControllercentric:
/\
||
||
_________ ________
| | | |
| igloo |<===\ |mchost|
|_______| \\ |______|
\\ ^
\\ |
\\ |
\\ |
_________ ||___|____
| | | |
(DC) | Boss |<------->|Prospero| (TC)
|_______| |________|
igloocentric:
/\
||
||
_________ ________
| |--------->| |
(TC) | igloo |<===\ |mchost|
|_______| \\ |______|
^ \\
| \\
| \\
| \\
____|____ ||________
| | | |
(DC) | Boss |<========|Prospero|
|_______| |________|
Modcompless:
/\
||
||
_________ ________
(DC) | | | |
(TC) | igloo |<-------->|mchost|
|_______| |______|
/\
||
||
||
|| __________
|| | |
\=============|Prospero|
|________|
More information about the evla-sw-discuss
mailing list