[evla-sw-discuss] Switched power calibration at the EVLA

[Michael Rupen]

Hi Steve --


> Since we observe with Tcal firing during (nominally half) the observing
> cycle, our proxy for Psys is (Pon + Poff)/2, e.g. the state count weights
> going into the Pon + Poff are the same as in the correlator accumulation
> of r_12 (which will see Tcal the same amount of time).  Is this assured? 
> Probably a small effect anyway.  Just checking.

I believe this is mostly assured, but KEN can correct me if I'm wrong. The 
ON/OFF measurements are made just before the requantizer gain, which should be
after any RFI blanking etc. After the filter chip they go to the output chip 
and on to the correlator board, and in principle data could be lost along the 
way.  After correlation one tends to lose entire lag frames.


> Note we are used to having correlated flux, not Tant units for VLA.  See
> below, but are we really going to T units not S?

We are going from raw visibility to temperature to flux density, rather
than doing it all in one go.  This removes most of the annoying
characteristics of the raw visibilities without forcing us to measure
many more quantities than we have to hand at the moment.


> Tcal:
> I am reluctant to put things in the SDM that are not properties of the data 
> but of the system which must be externally determined, e.g. the gain curves 
> (some early SDMs will have incorrect gain curves, we will not update the SDM) 
> and Tcal (same).  We will at least have to provide ways to override whats in 
> the SDM.


The idea is to store our current best guess as to Tcal with the SDM,
so we are guaranteed to have *something* with the data.  This ensures
we can use the described calibration scheme from day 1. In the 
end we may want to fetch Tcal(freq) from another archive; this requires some
archiving, but also some "smarts" in taking the best values for a given
observing time.

A similar consideration comes with the antenna positions.  We do measure
and update these at irregular intervals, and better values become available;
yet we store the best positions we have at the time, with the data, and
then allow the user to plug in something better if he has it.




>>    b. Pdiff== (Pon-Poff)/G and  Psum== (Pon+Poff)/G
>>      From these one can trivially calculate both Tsys and the visibility
>>      scaling factor 1/sqrt(Pdiff1*Pdiff2). We prefer to store Psum
>>      rather than Tcal/Tsys because that's closer to what's actually
>>      measured.
>
> For flux units
>       S_ij = r_ij*sqrt(CS_i*CS_j)
>           Correction factor CS_i = 2kTcal_i/(Pdiff_i*etacorr_i*Aeff_i)
>           *I think the factor of two is there in this case 
> For temperature units
>       Tant_ij = r_ij*sqrt(CT_i*CT_j)
>          Correction factor CT_i = Tcal_i/Pdiff_i
> For Tsys (e.g. for weight scaling, editing)
>       Tsys_i  = (Psum_i/Pdiff_i)*Tcal_i/2
>
> If wanted
>       rho_ij = r_ij*sqrt(CR_i*CR_j)
>           Correction factor CR_i = 2*Pdiff_i/Psum_i
>
> Note that to get to flux units, need the conversion (e.g. Aeff
> as a function of frequency) in addition.  I see nowhere in SDM for
> that so we will be looking this up (externally or internally) anyway.


Or we define a new table and store that as well.  I agree that this
is all poorly defined at the moment, and my tendency would be to add
a gain curve to the SDM as well.  I believe ALMA will have a separate 
Calibration Database and we could go that route, retrieving CalCurve Tables
as we go.  If the observatory is doing many of the routine calibrations
we have to have some way to provide those to users, either with the SDM
or via external retrieval.  This has not been thought out well enough
by any means.

With the current VLA we do apply a gain curve but fundamentally the
conversion to flux densities comes from comparing with a flux calibrator.
In AIPS this is done by most people in the filler.


>>    b. Pdiff & Psum
>>      The SysCal Table is the obvious place for these, but there are
>>      no obvious spots for them.  We could either define new optional data
>>      switchedPowerDifference[Nrec][Nchan] and switchedPowerSum[Nrec][Nchan]
>>      or re-define the Table.  I propose changing the Table definition
>>      as follows:
>>
>>        Key (no change):
>>          antennaId
>>          spectralWindowId
>>          timeInterval
>>          feedId
>>        Required data:
>>          numReceptor
>>          numChan
>>          calType  enumeration: on the EVLA side we need:
>>                            Tcal
>>                            switchedPowerDifference
>>                            switchedPowerSum
>>                            and possibly requantizerGain (see below)
>
> I dont understand why an enumeration - do you imply we have a separate row 
> for these?

Yes, if we stick to the SysCal Table.  I'm not very fond of the
infinite number of optional data in the current SysCal Table, though
I can see where that might be good for post-processing.

>>          calSpectrum  float[Nrec][Nchan]
>>
>>      and no optional data.  This seems a more obvious way to store these
>>      values than invoking one of the current large set of purely optional
>>      data, especially if the MS SysCal definition may be changing.
>
> I would prefer to define a new table SysPower with these (so not to confuse 
> with
> the MS SysCal table, which we will likely not redefine)


I'm happy either way.  The ALMA side seems to like to minimize the
number of tables, and certainly these pow* values are very close to the
SysCal type of information.  Note that the SysCal Table already has
data columns which are not temperature, e.g. Tant/Tsys and the phase
difference spectrum.  I'd rather have a more general table than two 
independent ones, esp. ones which are set independently by telescope.



>        Key:
>          antennaId
>          spectralWindowId
>          timeInterval
>          feedId
>        Required data:
>          numReceptor
>          switchedPowerDifference float[Nrec]
>          switchedPowerSum float[Nrec]
>          requantizerGain float[Nrec] (optional)
>
> (Note - this is not per channel but refers to the whole spw,
> so we no longer have to redefine new fake spw for the calibration info
> - yea! But if there is any reason that this will later become channelized
> within a spw then this would need to then be per channel.)


The only channelized version for us should be the autocorrelations, and
we already have a place for those (the BDF).

Note that the bandwidth and central frequency for the switched power
will not in general match those of the corresponding visibilities, if
for instance we throw away some visibility channels to save space,
or stitch together multiple subbands.  Also there is the case where we
have switched power measurements but no visibilities.  That will require
a separate SpW in any case, so you haven't gotten away from them altogether.


>>  7. What we do in post-processing (filler & beyond)
>>    Priorities are a la Claire, assuming we must have scaled visibilities
>>    for OSRO.  If not add 1 to all priorities.
>>    a. Interpolate Tcal to match the Tsys Spectral Windows
>>      This should be straightforward -- a linear interpolation would be
>>      fine.
>>
>>     Priority 0 for OSRO.
>
> I would prefer to define and store Tcal polynomials over each band and put
> in a database somewhere like gain curves and antenna positions.


Fine by me -- ideal in fact -- if we have such a data base by the time
we need it.  I'm not aware of any work in this area, but maybe BRYAN
can correct me.


>>    b. Smooth/average/flag Pdiff, Psum
>>      Some sort of averaging, and tossing of bad numbers, is probably
>>      needed.  The most important at first would be the tossing of bad
>>      numbers
>>      I suspect.
>>
>>     Priority 1 for OSRO (averaging may be essential for narrow subbands).
>
> I would refine a bit:
>
>     b. Compute scaling factors C_i, C_j for r_ij to user units (Tant, S)
>        These (in CASA) would be a new type of calibration table "C"
>        that could be manipulated like other cal tables (with
>        addition of carrying along C_i and Tsys_i
>
>        b1) populate with C_i = 2kTcal_i/(Pdiff_i*etacorr_i*Aeff_i) for flux
>            conversion, including interpolation over missing/edited values
>            (Priority 0)
>        b2) editing of solutions/smoothing/reinterpolation of C_i (Priority
>        1)


Fine by me.  I was trying to skirt implementation details in CASA since
I'm not really competent to discuss those.


>>    e. Scale visibilities r_12 to Tant using (smoothed) Pdiff,
>>      (interpolated) Tcal
>>
>>     Priority 0 for OSRO.
>
> I would say scaling to correlate flux Priority 0, to Tant priority 1.


Only if we have the requisite scaling factors lying around.  We do
have Tcal.

Cheers,

           Michael