[evla-sw-discuss] Switched power calibration at the EVLA

[Steven T. Myers]

On Wed, 9 Dec 2009, Michael Rupen wrote:

>    Given Pon, Poff, and Tcal, we have:
>
>         Pon + Poff               2 Tnocal + Tcal
>         ---------- * Tcal/2  =   --------------- * Tcal/2  =  Tsys
>         Pon - Poff                    Tcal

Since we observe with Tcal firing during (nominally half) the observing
cycle, our proxy for Psys is (Pon + Poff)/2, e.g. the state count weights
going into the Pon + Poff are the same as in the correlator accumulation
of r_12 (which will see Tcal the same amount of time).  Is this assured? 
Probably a small effect anyway.  Just checking.

>     Barry points out that the gains are quite stable compared to the
>     variable Tsys, so it makes sense to skip the division by the total
>     power.  Instead use
>
>         Tant = Tant/Tcal * Tcal
>              = r_12/sqrt(Pc1*Pc2) * sqrt(Tcal1*Tcal2)
>
>     Pc here is Pdiff= (Pon-Poff)/G, the observed post-requantizer response
>     to the noise tube Tcal.  We don't worry about the rapidly-changing Tsys,
>     but instead keep track of the slowly-changing gain (Pc) and Tcal.
>     The correlator writes out r directly, without any scaling; we store
>     (Pon-Poff)/G and Tcal in the SDM, and apply those in post-processing.
>     One worry is that TelCal may eventually need these scaled values (Tant)
>     but for most purposes (e.g., antsol & ref.ptg.) and certainly for OSRO
>     the raw r are sufficient.

Note we are used to having correlated flux, not Tant units for VLA.  See
below, but are we really going to T units not S?

> 3. What we should store in the BDF
>   a. Visibilities
>     We propose continuing to store r_12 rather than rho_12 in the BDF,
>     i.e., the raw spectra out of the correlator scaled by data valid.

I agree.  Note that post-processing we could of course provide the option
to convert back to corr.coeff.

   rho_12 = r_12 * sqrt(Pc1*Pc2) / sqrt(P1*P2)
          = r_12 * sqrt(Pdiff1*Pdiff2) * 2/sqrt(Psum1*Psum2)

>   b. Data weights
>     Data weights should go as BW*tau/(Tsys_i*Tsys_j).  The CBE has
>     easy access to enough information to calculate BW*tau (note that
>     tau accounts for the data valid count), so we propose storing
>         BW*tau
>     as the data weight coming out of the correlator.

Seems to be the most straightforward thing to store.

> 5. What we should store in the SDM
>   a. Tcal vs. frequency
>     Barry suggests this be done on-line, so we store Tcal(SpWin) in the SDM;
>     Steve & Michael were happy with CASA handling this, treating it as
>     analogous to the antenna gain vs. elevation where those curves are
>     stored directly and then evaluated for the observed elevations as a
>     separate step in the calibration process.  George pointed out that more
>     steps means more like VLBI means unhappy & confused observers.
>
>     There will be one set of Tcal per Scheduling Block -- actually they'll
>     change even more slowly.
>
>     Conclusion: for now store Tcal at the engineer-measured frequencies
>       with each SDM.  MCAF will have to fetch this info from some data base
>       but there aren't that many numbers to store.
>
>    Michael will look into where this should go in the SDM.

I am reluctant to put things in the SDM that are not properties of the data but 
of the system which must be externally determined, e.g. the gain curves (some 
early SDMs will have incorrect gain curves, we will not update the SDM) and Tcal 
(same).  We will at least have to provide ways to override whats in the SDM.

>   b. Pdiff== (Pon-Poff)/G and  Psum== (Pon+Poff)/G
>     From these one can trivially calculate both Tsys and the visibility
>     scaling factor 1/sqrt(Pdiff1*Pdiff2). We prefer to store Psum
>     rather than Tcal/Tsys because that's closer to what's actually
>     measured.

For flux units
       S_ij = r_ij*sqrt(CS_i*CS_j)
           Correction factor CS_i = 2kTcal_i/(Pdiff_i*etacorr_i*Aeff_i)
           *I think the factor of two is there in this case 
For temperature units
       Tant_ij = r_ij*sqrt(CT_i*CT_j)
           Correction factor CT_i = Tcal_i/Pdiff_i
For Tsys (e.g. for weight scaling, editing)
       Tsys_i  = (Psum_i/Pdiff_i)*Tcal_i/2

If wanted
       rho_ij = r_ij*sqrt(CR_i*CR_j)
           Correction factor CR_i = 2*Pdiff_i/Psum_i

Note that to get to flux units, need the conversion (e.g. Aeff
as a function of frequency) in addition.  I see nowhere in SDM for
that so we will be looking this up (externally or internally) anyway.

>     Eventually we will want to store G whenever it changes, primarily
>     (only?) to allow the stitching together of subbands in certain
>     cases.  If we want to do such stitching in real time that would
>     require sending G to the CBE.

Fair enough, could store in SDM.

> 6. Implications for the SDM
>   a. Tcal vs. frequency
>     The most obvious place to store this would be the CalDevice Table,
>     which is intended to store calibration device characteristics, and
>     includes the equivalent temperatures of the noise sources (noiseCal) as
>     an optional datum.  Unfortunately these are supposed to be independent
>     of frequency within a Spectral Window.  We could either interpolate
>     to the Tsys Spectral Windows in real time, or introduce a bunch of
>     (several hundred) SpW just for Tcal.  This seems rather excessive.
>     The alternative would be to modify the CalDevice Table to allow
>     storing spectra, which would be a fairly major change.

Yes, this seems to be for loads.

>     The SysCal Table purports to "give information on the conversion of
>     data to temperature scale", and has tcalSpectrum as one of its optional
>     data.  If we don't mind overloading this Table this seems the most
>     obvious place to store Tcal for now.

I think this is the correct use of this table, though again I would prefer
to not put this in the SDM at all.  But if you do, this is probably the
place, and maybe could consider storing a polynomial over the spw instead
of per channel (since you would likely be interpolating from a polynomial
anyway).

Also, what about antenna efficiencies etc for flux scaling?

>   b. Pdiff & Psum
>     The SysCal Table is the obvious place for these, but there are
>     no obvious spots for them.  We could either define new optional data
>     switchedPowerDifference[Nrec][Nchan] and switchedPowerSum[Nrec][Nchan]
>     or re-define the Table.  I propose changing the Table definition
>     as follows:
>
>       Key (no change):
>         antennaId
>         spectralWindowId
>         timeInterval
>         feedId
>       Required data:
>         numReceptor
>         numChan
>         calType  enumeration: on the EVLA side we need:
>                           Tcal
>                           switchedPowerDifference
>                           switchedPowerSum
>                           and possibly requantizerGain (see below)

I dont understand why an enumeration - do you imply we have a separate row for 
these?

>         calSpectrum  float[Nrec][Nchan]
>
>     and no optional data.  This seems a more obvious way to store these
>     values than invoking one of the current large set of purely optional
>     data, especially if the MS SysCal definition may be changing.

I would prefer to define a new table SysPower with these (so not to confuse with
the MS SysCal table, which we will likely not redefine)

        Key:
          antennaId
          spectralWindowId
          timeInterval
          feedId
        Required data:
          numReceptor
          switchedPowerDifference float[Nrec]
          switchedPowerSum float[Nrec]
          requantizerGain float[Nrec] (optional)

(Note - this is not per channel but refers to the whole spw,
so we no longer have to redefine new fake spw for the calibration info
- yea! But if there is any reason that this will later become channelized
within a spw then this would need to then be per channel.)

Then in the SysCal table put the
          tcalSpectrum  float[Nrec][Nchan]
with a possible addition of
          numTerm
          tcalSpectrumPoly  float[Nrec][Nterm] (optional)
if you want to save polynomials.

(Note - this is in the actual nchan of the spw as was intended for SysCal
and will usually be a single time interval for the whole observation. Set
all other quantities in this table to unity.)

>     Note that we are using separate (associated) Spectral Windows for
>     the visibilities and their corresponding calibration values.

Not under the above scheme.  The spwid in the SysPower table tells it
which spw it refers to.

> 7. What we do in post-processing (filler & beyond)
>   Priorities are a la Claire, assuming we must have scaled visibilities
>   for OSRO.  If not add 1 to all priorities.
>   a. Interpolate Tcal to match the Tsys Spectral Windows
>     This should be straightforward -- a linear interpolation would be fine.
>
>    Priority 0 for OSRO.

I would prefer to define and store Tcal polynomials over each band and put
in a database somewhere like gain curves and antenna positions.

>   b. Smooth/average/flag Pdiff, Psum
>     Some sort of averaging, and tossing of bad numbers, is probably
>     needed.  The most important at first would be the tossing of bad numbers
>     I suspect.
>
>    Priority 1 for OSRO (averaging may be essential for narrow subbands).

I would refine a bit:

     b. Compute scaling factors C_i, C_j for r_ij to user units (Tant, S)
        These (in CASA) would be a new type of calibration table "C"
        that could be manipulated like other cal tables (with
        addition of carrying along C_i and Tsys_i

        b1) populate with C_i = 2kTcal_i/(Pdiff_i*etacorr_i*Aeff_i) for flux
            conversion, including interpolation over missing/edited values
            (Priority 0)
        b2) editing of solutions/smoothing/reinterpolation of C_i (Priority 1)

>   d. Calculate Tsys based on (possibly smoothed) Pdiff, Psum,
>     (interpolated) Tcal
>    :  May want to average/smooth/flag Tsys
>
>    Priority 2 for OSRO (Tsys not absolutely required to calibrate
>       visibilities)

Would do same operation along with b2.

>   e. Scale visibilities r_12 to Tant using (smoothed) Pdiff,
>     (interpolated) Tcal
>
>    Priority 0 for OSRO.

I would say scaling to correlate flux Priority 0, to Tant priority 1.

>   i. Export to AIPS UVFITS
>    :  Still need to discuss the details of what is essential here.
>
>    Priority 0 for OSRO

Export what?  Export corrected data or export some sort of TY table?

>   b. Steve suggests Tcal be treated like antenna gains in being a set of
>     calibration numbers which are not sampled like the data, but which
>     CASA has to interpolate onto the observations.  Michael likes this idea;
>     Barry thinks we might as well do the interpolation on-line.

Yes, see above. Also eta*Aeff to get fluxes if we want that capability.

>  b. What shouild be sent to AIPS via exportuvfits?

indeed.