[evla-sw-discuss] Switched power calibration at the EVLA
Michael Rupen
mrupen at nrao.edu
Wed Dec 9 09:04:42 EST 2009
EVLA Calibration: uses of switched power measurements
=====================================================
mrupen 30nov09
mrupen 8dec09: modified per George's comments (esp. definition of
weights!)
Minutes & results of 30nov09 discussion, based on Vivek's 27nov09 e-mail
Present: bclark, nelias, nlorente, gmoellen, rmoeser, smyers, rperley,
mpokorny, mrupen, ksowinsk
1. EVLA hardware basics
There are 128 Station Boards -- 4 StB for each of 32 antennas (we only
use 27x4= 108 StB normally).
Each Station Board has 2x(16+2)= 38 filter chips, defining 38 subbands
with bandwidths from 31.25 kHz to 128 MHz.
Within each filter chip there are...
- a synchronous power detector after all the filtering but before the
requantizer, so you have access to lots of bits. This gives us:
Pon= (Power when noise tube is on) /(counts when noise tube is on)
Poff= (Power when noise tube is off)/(counts when noise tube is off)
- a requantizer, which takes the 16 bits out of the filters, multiplies
by some settable factor G (the requantizer gain), and resamples to
either 4- or 7-bits
- another power meter after the requantizer, which is NOT synchronized
to the noise tube -- it just gives a single power, PRQP
(post-requantizer power).
- a state counter, also just after the requantizer, which counts
the population in each quantum state in some time-multiplexed (round
robin) fashion
The requantized signals from each filter chip are sent on to the
correlator chips for correlation.
Graphically:
(Pon, Poff)
SynchPowerDetector PRQP
^ ^
| |
filters ----> Requantizer -----> correlation
(gain G) |
V
state
counter
The fundamental correlator "heartbeat" is 10 msec, and the hardware
reports the power measurments every 10 msec.
Currently the full noise tube cycle is 19.2 Hz. There are plans to change
this to some even multiple of 1/10 msec.
a. ALMA does not have noise tubes, and hence does not continuously
measure gain or Tsys. Instead ALMA relies on occasional (maybe
half-hour-ish?) Tsys measurements using cold & hot loads.
2. What we want from these power measurements
a. Correct setting of the requantizer gain to maximize SNR.
For this we use the PRQP, setting G so that the rms PRQP is the optimal
value (2.7).
- The idea is that G will be set infrequently, at the command of the
observing script. For OSRO we hope "infrequently" means "set to
a certain value for a given receiver band" -- set-and-forget,
basically -- and we may even be able to get away with a single
default setting for all bands.
- The only obvious reason to change G is to get somewhat better SNR.
This does not seem a huge issue at the moment.
- This is the ONLY use for the PQRP.
Thus we would like to store G (which changes very slowly) in the SDM,
but not the PRQP. The correlator will internally use the PRQP to
figure out G in real time but there's no need to save the PRQP for
later perusal.
We don't need to store G at first but will want it in the long run
(e.g., for subband stitching), so we should probably just bite the
bullet and find a place for it in the SDM now.
b. Tsys in Kelvins.
Tsys is useful in three ways:
(i) in deriving data weights (rms goes as Tsys/sqrt(BW*tau), so
data weights go as 1/rms^2= BW*tau/(Tsys_i*Tsys_j) ) ;
(ii) in deriving the opacity (this requires real kelvins) ; and
(iii) as a diagnostic -- finding lousy antennas, flagging
rapidly-variable RFI which dominates the subband Tsys, etc.
Given Pon, Poff, and Tcal, we have:
Pon + Poff 2 Tnocal + Tcal
---------- * Tcal/2 = --------------- * Tcal/2 = Tsys
Pon - Poff Tcal
We would like to record this as often as it changes, which could be fast
in the RFI case or if a receiver has some instability.
Note that Tsys is not directly needed for scaling the visibilities,
but *is* needed for scaling the data weights.
- Note that we may wish to measure Tsys using a wider bandwidth than
used for the visibilities -- WIDAR can tune two subbands with
different bandwidths to the same central observing frequency, for
instance, and this may be useful for narrow subbands.
c. Scaling the visibilities to (at least an approximation of) Kelvin --
i.e., taking out gain changes between the introduction of the noise tube
and the measurement of the switched power, which covers basically
everything after the radiation is collected in the feed.
The correlator chips produce the correlation r.
- This does not change e.g. when a cloud passes in front of a telescope
(ignoring absorption) -- the correlator just measures the correlated
power, which does not depend on the system temperature (i.e., the
total power).
Traditionally in the past we've normalized by the total power,
recording what AIPS has called the correlation coefficients (sorry
Barry):
rho_12= r_12/sqrt(P_1*P_2)
This is basically Tant/Tsys, and in the filler we multiply this by Tsys to
get observed Kelvins:
Tant= Tant/Tsys * Tsys
= r_12/sqrt(P1*P2) * sqrt(Tsys1*Tsys2)
P here is (Pon+Poff)/(2 G), the observed post-requantizer response to
the sky power Tsys.
[Currently with FILLM we also multiply by a fudge factor for the
antenna gain but that's really a separate correction, not considered
here.]
Barry points out that the gains are quite stable compared to the
variable Tsys, so it makes sense to skip the division by the total
power. Instead use
Tant = Tant/Tcal * Tcal
= r_12/sqrt(Pc1*Pc2) * sqrt(Tcal1*Tcal2)
Pc here is Pdiff= (Pon-Poff)/G, the observed post-requantizer response to
the noise tube Tcal. We don't worry about the rapidly-changing Tsys,
but instead keep track of the slowly-changing gain (Pc) and Tcal.
The correlator writes out r directly, without any scaling; we store
(Pon-Poff)/G and Tcal in the SDM, and apply those in post-processing.
One worry is that TelCal may eventually need these scaled values (Tant)
but for most purposes (e.g., antsol & ref.ptg.) and certainly for OSRO
the raw r are sufficient.
- Note that we may wish to measure Pc using a wider bandwidth than
used for the visibilities -- WIDAR can tune two subbands with
different bandwidths to the same central observing frequency, for
instance, and this may be useful for narrow subbands.
3. What we should store in the BDF
a. Visibilities
We propose continuing to store r_12 rather than rho_12 in the BDF,
i.e., the raw spectra out of the correlator scaled by data valid.
b. Data weights
Data weights should go as BW*tau/(Tsys_i*Tsys_j). The CBE has
easy access to enough information to calculate BW*tau (note that
tau accounts for the data valid count), so we propose storing
BW*tau
as the data weight coming out of the correlator.
- Michael and Ken must provide Martin with the requisite formula,
which is a function of the data valid count and the filter width.
- When chaining together multiple correlator chips or using
recirculation the data valid count may vary with the lag frames which
go into a lag set. In principle we could account for this by tracking
the corresponding variation in data weight with channel (i.e., the
Fourier transform of the data valid count vs. lag), but that seems
excessive, certainly in the near term. Instead I propose using the
average data valid count for all the lag frames which go into a given
lag set. Either the effect is minor & this will be good enough,
or the effect is major and will show up as a significant drop in this
average weight, which can be used to flag the corresponding
visibilities. Note that an entirely missing lag frame should lead
to the corresponding lag set being dropped.
4. Implications for the BDF
The current BDF can handle what we want, since all the interesting
ancillary/meta-data are stored in the SDM.
5. What we should store in the SDM
a. Tcal vs. frequency
These come from the engineers via some parameters data base. From
Vivek's 27nov09 e-mail:
Tcal is measured on the bench every 25 MHz at L & S bands
every 50 MHz at C band
every 100 MHz at X & higher bands.
Total then is ~1000 numbers x 2 pol'ns x 32 receivers.
In general the frequencies at which Tcal was measured will not match the
observed Spectral Windows, so we need to interpolate somehow.
Barry suggests this be done on-line, so we store Tcal(SpWin) in the SDM;
Steve & Michael were happy with CASA handling this, treating it as
analogous to the antenna gain vs. elevation where those curves are
stored directly and then evaluated for the observed elevations as a
separate step in the calibration process. George pointed out that more
steps means more like VLBI means unhappy & confused observers.
There will be one set of Tcal per Scheduling Block -- actually they'll
change even more slowly.
Conclusion: for now store Tcal at the engineer-measured frequencies
with each SDM. MCAF will have to fetch this info from some data base
but there aren't that many numbers to store.
Michael will look into where this should go in the SDM.
b. Pdiff== (Pon-Poff)/G and Psum== (Pon+Poff)/G
From these one can trivially calculate both Tsys and the visibility
scaling factor 1/sqrt(Pdiff1*Pdiff2). We prefer to store Psum
rather than Tcal/Tsys because that's closer to what's actually
measured.
These are usefully stored at most once per noise tube period, i.e.,
1/19.2 Hz~ 50 msec currently. We will likely start by storing one
number per integration or per second.
Ken, as modified by Vivek, says:
"I suggest that the integration interval be a multiple of the 10 msec
correlator tick, that they be aligned with the tick, and, for
simplicity, that the number of integration ticks evenly divide a
day and that an integration tick occur exactly at midnight.
Integration intervals less than 10 msec can be accomodated if IO
rates allow but the normalization will always be based on at least
1/19.2 Hz of total power measurements."
c. G (requantizer gain)
Eventually we will want to store G whenever it changes, primarily
(only?) to allow the stitching together of subbands in certain
cases. If we want to do such stitching in real time that would
require sending G to the CBE.
Our current thinking is that G will change only occasionally,
when commanded by the observing script, and may even remain
constant for an entire Exec Block & beyond.
6. Implications for the SDM
a. Tcal vs. frequency
The most obvious place to store this would be the CalDevice Table,
which is intended to store calibration device characteristics, and
includes the equivalent temperatures of the noise sources (noiseCal) as
an optional datum. Unfortunately these are supposed to be independent
of frequency within a Spectral Window. We could either interpolate
to the Tsys Spectral Windows in real time, or introduce a bunch of
(several hundred) SpW just for Tcal. This seems rather excessive.
The alternative would be to modify the CalDevice Table to allow
storing spectra, which would be a fairly major change.
The SysCal Table purports to "give information on the conversion of
data to temperature scale", and has tcalSpectrum as one of its optional
data. If we don't mind overloading this Table this seems the most
obvious place to store Tcal for now.
b. Pdiff & Psum
The SysCal Table is the obvious place for these, but there are
no obvious spots for them. We could either define new optional data
switchedPowerDifference[Nrec][Nchan] and switchedPowerSum[Nrec][Nchan]
or re-define the Table. I propose changing the Table definition
as follows:
Key (no change):
antennaId
spectralWindowId
timeInterval
feedId
Required data:
numReceptor
numChan
calType enumeration: on the EVLA side we need:
Tcal
switchedPowerDifference
switchedPowerSum
and possibly requantizerGain (see below)
calSpectrum float[Nrec][Nchan]
and no optional data. This seems a more obvious way to store these
values than invoking one of the current large set of purely optional
data, especially if the MS SysCal definition may be changing.
Note that we are using separate (associated) Spectral Windows for
the visibilities and their corresponding calibration values.
- For the EVLA:
Nrec= 1 or 2 (R, L)
Tcal --> Nchan= few thousand
switchedPower* --> many SpW, but each has Nchan=1
This potentially leads to very large SysCal Tables: in the long run,
of order
1 row/filter * 38 filters/StB * 128 StB= 4864 rows per dump
8 hours * 1 dump/50 msec * 4864 rows/dump = 2.8 billion rows
Even with 1sec dumps we get 140 million rows in 8 hours, with
each row having 2 floating point numbers plus overhead.
- For OSRO the SysCal Table will be much more modest:
1 row/filter * 2 filters/StB * 128 StB= 256 rows per dump
8 hours * 1 dump/1 sec * 256 rows/dump = 7.4 million rows
c. G
The SysCal Table seems a good place for this, if modified as above.
The data rate for G is trivial.
7. What we do in post-processing (filler & beyond)
Priorities are a la Claire, assuming we must have scaled visibilities
for OSRO. If not add 1 to all priorities.
a. Interpolate Tcal to match the Tsys Spectral Windows
This should be straightforward -- a linear interpolation would be fine.
Priority 0 for OSRO.
b. Smooth/average/flag Pdiff, Psum
Some sort of averaging, and tossing of bad numbers, is probably
needed. The most important at first would be the tossing of bad numbers
I suspect.
Priority 1 for OSRO (averaging may be essential for narrow subbands).
c. Ability to apply Pdiff, Psum measured in one subband to
the data taken in another
Not needed for OSRO.
d. Calculate Tsys based on (possibly smoothed) Pdiff, Psum,
(interpolated) Tcal
: May want to average/smooth/flag Tsys
Priority 2 for OSRO (Tsys not absolutely required to calibrate
visibilities)
e. Scale visibilities r_12 to Tant using (smoothed) Pdiff,
(interpolated) Tcal
Priority 0 for OSRO.
f. Scale correlator weights by (smoothed) Tsys
Priority 2 for OSRO (we lived with uniform weights for decades)
g. Plot all of the above vs. time, frequency (Pdiff, Psum, Tcal,
Tsys, visibilities, weights; G)
Priority 1 for OSRO
h. Flag visibilities based on any of the above (Pdiff, Psum Tsys,
weights, G)
Priority 2 for OSRO
i. Export to AIPS UVFITS
: Still need to discuss the details of what is essential here.
Priority 0 for OSRO
8. Implications for post-processing
a. Ideally the data weights should be in the same units as the
visibilities. This is not required for standard imaging but
is essential for calculating uncertainties (error bars) --
basically the difference between finding a minimum chi-squared,
and knowing what that minimum value means.
Priority 2 for OSRO alas
b. George points out that we should get the visibilities & their
weights on the same scale ASAP, preferably by the time we have a
Measurement Set. This requires the filler to figure out Tcal & Tsys
so smoothing becomes more painful, as does flagging bad Tcal & Tsys.
We would like to be able to undo and re-do any such calculation if
possible, though I'd call that
Priority 2 for OSRO.
On the plus side, Francois has already asked about any modifications to
the DAMs (data access methods) for this purpose.
On the minus side, the more we put on the filler, the more carefully
we have to monitor (and influence) what the filler actually does.
MPR: My own preference is to keep a close eye on what's done until
we've played with the actual data and figured out what to do -- i.e.,
keep this in CASA for now.
b. Steve suggests Tcal be treated like antenna gains in being a set of
calibration numbers which are not sampled like the data, but which
CASA has to interpolate onto the observations. Michael likes this idea;
Barry thinks we might as well do the interpolation on-line.
9. Misc.
a. The underlying assumptions in much of the above are that (1) Tcal
is more stable than gain, and (2) gain is more stable than Tsys.
Rick wonders why we bother with a fast noise tube at all, if the
gains are so stable. For that matter, why not turn the NT off most of
the time, and get a couple kelvins of Tsys back?
MPR: My personal feeling is that we stick with our current approach
(NT always switching) because it's familiar, we understand it, we
know it will work, and it catches many things we might not think
through carefully enough in the press of getting to OSRO.
b. This note does not consider contributions to gain upstream
of Tcal injection, which must be handed separately: opacity,
gain curve (Jy/K), pointing, focus, etc. We also did not discuss
Van Vleck corrections, subband stitching, the Sun, pulsars, etc.
10. Remaining questions
a. Pdiff and Psum must be averaged down to the requisite time interval
on the Station Board.
- Must these averages be in synch with the visibility dumps?
b. What shouild be sent to AIPS via exportuvfits?
===============================================================================
More information about the evla-sw-discuss
mailing list