[daip] [Fwd: Re: Question about DELZN]

[Leonia Kogan]

Hi Mark,

May I ask to say your opinion about the following message. In particular 
Richard Dodson ir writing:


As an aside the correlator output of a phase referencing experiment, of
course, does not include the geometry (of the target) as that is what is
being sought. This does not therefore fit your requirements. This
certainly effects the Sgr A* work of Mark Reid and the similar VERA
projects. I take this as meaning that Mark does not use DELZN?

                                                                            
^^^^^^^^^^^^^^^^^^^^^^^
You do not use DELZN.   Do you?

Do you think that DELZN can work using only one calibrator? I ve never 
tried it .

Thank you very much in advance

Leonia


-------- Original Message --------
Subject: 	Re: Question about DELZN
Date: 	03 Mar 2006 12:15:08 +0100
From: 	Richard Dodson <r.dodson at oan.es>
To: 	Leonia Kogan <lkogan at nrao.edu>
CC: 	daip at nrao.edu, maria rioja <mj.rioja at oan.es>, r.dodson at oan.es, 
amiodusz at nrao.edu
References: 
<e8b6f9810602280447x330222e2s238265dff38cce93 at mail.gmail.com> 
<4405DEAA.8010306 at nrao.edu>



Hi Leonia, 

> Let me explain you  what and how DELZN is doing.

Thanks

> Lets note that VAL_IANT -VAL_RANT are the values picked up from the SN
> table as a function of time.
> It can be multi band delay phase or ....
> It is always supposed that the VALs are residuals.

Of course. But residuals can still contain several several
contributions. 

> THE MAIN POSSIBLE ERRORS (such a geometry, initial model of the
> atmosphere and clocks and....)  ARE COMPENSATED DURING CORRELATION.
> THEREFORE  WE CAN CONSIDER THE VALs INCLUDES ONLY THE EFFECT OF THE
> UNCOMPENSATED ATMOSPHERE AND CLOCKS.

OK. As I said, we are pushing DELZN. We don't have 500 MHz of bandwidth,
nor multiple sources. In many ways we are not conforming to the defined
paths. However one can not do new things on the defined paths, and new
instruments can't necessarily follow them. I think by leaving the
defined paths we are uncovering some new insights. 

As an aside the correlator output of a phase referencing experiment, of
course, does not include the geometry (of the target) as that is what is
being sought. This does not therefore fit your requirements. This
certainly effects the Sgr A* work of Mark Reid and the similar VERA
projects. I take this as meaning that Mark does not use DELZN?

Nevertheless, I continue. 

> Additionally  DELZN considers that the atmosphere behaviour is 
> described as the function of senith angle elevation through
> the mapping function as COSEC.

Yes (although we should watch out for the sec(Z)tan(Z)\Delta Z mapping
function that is required for the phase referencing case, a la Mark
Reid. It maybe worth adding this mapping function?). However, as you say
DELZN will not fit for geometry (and I assume is not planned to do so?
It would be some, but not huge, work to add such a shift to the
minimisation. In this cases the clocks are zeroed so there are fewer
parameters), so it is not really suitable for phase referenced images.
Therefore the manner of its current use for VERA analysis is dubious,
but should not fail so catastrophically. 

Furthermore I believe it should be able to produce a sensible answer for
the atmosphere from the reference source alone as it does fit
tau_atmosphere as a function of time and as a time varying polynomial. 

> The SN table VALs are the difference of  antenna ANT and reference
> antenna RANT. For simplicity lets consider that the reference antenna
> 
> Therefore the the SN table values can be described by the following
> equation:
> 
> VAL_IANT) - VAL_RANT = ZENATM_IANT(time) *MAPFUNC(el)+
> CLOCK_IANT(time)        (1)
>                                                 -ZENATM_RANT(time)
> *MAPFUNC(el)
> DELZN fits polynomials to the   ZENATM_IANT,  ZENATM_RANT    and
> CLOCK_IANT
> 
> Actually CLOCK_IANT = CLOCK_IANT - CLOCK_RANT
> 
> So (for example) if the third order of polynomial (a*t+b*t^2+c) is
used
> for both  ZENATM_IANT and  CLOCK_IANT
> then 6 (six) parameters should be fitted for each antenna. 3 for atm
and
> 3 for clocks
> You wrote:
> 
> >  I believe there are 5 parameters to be fitted per antenna (for 3rd
> >  order polynomial for tau_a) (20 for VERA)
> 
> I do not know why 5 and what is it (20 for VERA)?

The help file for DELZN recommends a linear polynomial for the clocks
(aparm(3)) -- that is why I have 5 not 6 parameters per antenna. 4
antenna gives 20 parameters, but of course the reference antenna clocks
(as you point out) are not fitted, so 18. For the phase-only case the
clocks are solved for on the reference source, so there are few
parameters again. 

> I hope this message will help. 

 It does, but does not answer the bigger questions. I know there are a
lot of them! These are the most important I think, so I will repeat them
;): 

 I believe that (for some reason) the minimization in DELZN is failing.
Proof positive of this is the behaviour as one alters the number of
parameters in the fit. The plot is showing VAL(IANT-RANT)+ATM(RANT)
(clocks are zero). The data is shown scattered around the model values
for ATM. ATM is what we get in the text file with OUTFILE set, which is
what is shown in the plots I attached previous. If one alters the number
of parameters fitted for ATM one gets wildly different answers.
Therefore I am sure that the minimisation for ATM is misbehaving. 

 I agree that observations of multiple sources should/would avoid this
problem. However I can not see why multiple observations of one source
(with a range of elevations from 0 to 80) would not allow a solution for
the atmosphere, particularly if you allow for a timing varying
atmosphere.

 Can you tell me if there is fundamental problem with such a approach?
Has it been tried? By who and did they have any comments?

 I do not want to have to extract the data from AIPS and try fitting
externally, as this nullifies the point of what we are trying to achieve
(i.e. astrometric reduction of VERA data within AIPS). Furthermore
Honma-san has already succeeded in doing this. 

 I think I now understand what the plots are showing, but I would still
like to see what an expected plot might show. (I would expect a roughly
constant atmosphere with, perhaps, some linear trend.)

 The ATM polynomial is fitted against time, which is what one would
expect. It is therefore the fitting that is problematic. What
minimization method do you use? I find leasqr mentioned, but can not
find where it is called from -- this is a bog standard matrix inversion
and so one needs treat it with care. 

In particular (From the code)
C     Notes:
C       1) Strictly speaking, the design matrix will usually contain
C          rows of zeroes and therefore be singular.  This arises if no
C          observations sensitive to a particular parameter have been
C          done.
C             In practice, any such singularities are ignored and the
C          associated parameters remain undetermined.

 I have a gut instinct this is where we are hitting the problem. 

 And perhaps most importantly -- as it implies mis-use of DELZN can
bugger up the data reduction -- do you think that DELZN as we are using
it is just performing a selfcalibration? 




 I have some suggestions: 

a) DELZN could possibly be used for the atmosphere correction from the
phase of the CALIBed _target source_, if the mapping function was
sec(z)tan(z)\Delta Z, rather than sec(z) as currently. The extra terms
are smoothly varying, so a polynomial fit for a (constant) atmosphere
could follow them -- however high order polynomials imply more unknowns
than freedoms. 
 I could add this new mapping function. 

 But one would really want to solve for geometry at the same time -- a
simple offset from the centre is all that is required. In this current
approach the geometrical offset is _assumed_ to be identifiable in the
dirty map. This was not the case for Sgr A* so may be a poor assumption.

 Addition of the geometrical terms would allow DELZN to be used for the
phase residuals of phase referencing experiments. 
 
b) DELZN should work for the reference source only (i.e. a single
source) with MB delays. There are a more unknowns (~5 per antennae) than
observables (~the number of antennae) per time interval, but assuming
solutions are independent (=significantly different elevation?) on a
reasonably short (1 hour say?) time interval this should not be a
problem. Has anyone tried DELZN for a single source with the VLBA? 

 As I said previously, this problem is hard to explain. 


      Richard



> Richard Dodson wrote:
> 
> >Hi Amy, Leonia,
> >
> >Thanks for your quick answer to Maria and my questions. We have more.
> >
> >Also I started this email last week -- so somethings have been
> >answered. I hope my re-writing has not introduced any incoherency.
> >
> >
> >
> >>In an ideal world, what you did would work fine.  The problem is
that
> >>128MHz will, in most cases, be too small a bandwidth to obtain an
> >>accurate multiband delay.  As you see in Figure 2 of Memo 110 a
narrow
> >>bandwidth can easily give errors of a few nsec, which is why your
> >>multiband delays are large and give you nonsensical results from
DELZN.
> >>
> >>There is basically no way to fix your data, next time you have to
> >>observe with a wider bandwidth.
> >>
> >>
> >
> >  With the VERA setup that is not possible, or more accurately not
> >easy, to implement at this time. Therefore we are still looking to
> >find some way to apply the correction. You will have to be patient
> >with us, as we are stretching DELZN in ways that were never imagined
> >or intended.
> >
> >  What may be possible, or even necessary, is to introduce a short
> >section to the VERA schedules with scans of multiple known
> >calibrators. Currently there is a single scan of a fringe finder,
then
> >the dual beams track (continuously) the reference and the target
> >sources.
> >
> >  We are pretty confident, however, that the delay values should good
> >enough to support this analysis. In fact it is being done at the
> >moment (by others), by exporting the data and fitting externally.
What
> >we are trying to find is a route which is within AIPS.
> >
> >  128 MHz is only a factor of 4 less than your 500 MHz. The _scatter_
> >in the solutions is less than 1 nsec (typically 0.3). The solutions
of
> >course include clock offsets and rates, so the actual values are much
> >greater than this. However, if we have managed to understand your
> >memo, this can be accounted for.
> >
> >  However, where VERA will run into problems is the number of
freedoms
> >with the fitting routines. Recall that VERA has only 4 antennae, thus
> >only 3 MultiBand delays.  VERA has H-masers clocks,  but my
experience
> >with the Mitaka correlator (with VSOP data I admit) is that they
don't
> >usually bother to (or need to) produce perfect clock models for the
> >correlation run, so there could be considerable offsets and non-zero
> >rates.
> >
> >  Non-zero delays are also common with the VLBA. How have you removed
> >these in your analysis? Or does the multi-band delay fitting mean the
> >delays in the multiband column are somehow smaller than those in the
> >single band delays. Memo 110 does not make this clear (to me ;).
> >
> > The values for the atmosphere we derive from the MB delays are very
> >silly. The depth calculated is mega-millimeters! Now some questions.
> >
> >- Could you show us an example of a successful Atmospheric Zenith
Depth plot?
> >- When you say in the help file "a polynomial of the chosen degree
for
> >the zenith atmosphere delay" for the fitting against Zenith. Is this
> >the fit to the point by point solution? I.e. the variation of delay
> >with time, or is there a polynomial mapping function you are using?
> >- What might have happened to destroy the fitting algorithm?
> >
> > Basically I am suspicious about the numbers of parameters in the fit
> >and the consequences of it.
> >
> > You must be fitting
> >
> >\Delta \tau_{ij} = \tau_{atmos_i} sec Z_i - \tau_{atmos_j} sec Z_j +
> >                                    \tau_{clock_i}  - \tau_{clock_j}
> >
> >  Where \Delta \tau is the MBD (to reference j) and \tau_{clock} is
> >the linearly varying clock and \tau_{atmos} is the polynomially (in
> >time?) varying atmospheric delay. Z is the zenith angle.
> >
> > So plotting against elevation (fig4) is unlikely to show much unless
> >one assumes that the clocks are zero, and the plot is dominated by
the
> >effects from one antenna?
> >
> >  I believe there are 5 parameters to be fitted per antenna (for 3rd
> >order polynomial for tau_a) (20 for VERA). There are {the number of
> >antenna less one} delays per source and solution interval (3 per sol
> >interval for VERA). This makes it clear why multiple sources should
be
> >scheduled. Depending on how the solution interval is defined and used
> >it maybe possible to find solutions for our case. One would imagine
> >that if one had (say) 7 independent solution intervals over the
> >experiment one would be over determined for both of these kinds of
> >delay.
> >
> > This is what I would expect, but this is what would appear to not be
happening.
> >
> > We will experiment with trying to merge the calibrators into one
file
> >-- the sources are co-observed so have common antennae and time
stamps
> >-- but have not done this yet.
> >
> >  Does this make sense? I find it hard to express clearly. But there
> >is no doubt that we would prefer to use the cleaner MDEL solution as
> >the phase solution is a can of worms we don't want to open. But we
> >have.
> >
> >
> >  Part 2. The Phase based corrections
> >
> >
> >
> >>I have never used the phase so I cannot tell you how accurate it
should be
> >>
> >>
> >Oh dear. Is there some one we can ask about these? Mark Reid perhaps?
> >I am aware that these questions are beyond the usual scope.
> >
> >  Maria has successfully generated a DELZN solution from the CALIB
> >phases, but this is (possibly) contaminated by the dual beam phase
> >referencing approach used by VERA. It is important in this VERA
> >check-out-phase to ensure that this is not the case. In particular we
> >want to be sure that the expression used by M. Reid for the Sgr A*
> >analysis is that used. I.e. for the _differential_ zenith delay. They
> >were using the phases from the target source to calculate both the
> >atmospheric corrections and the source offset from zero. The
> >differential atmosphere delay correction is a function of \tau_0
> >sec(Z) tan(Z) \Delta Z. (ApJ 524 p816). This is a very similar
problem
> >and you suggest in memo 110 that Mark `initiated' the DELZN approach.
> >Does he use DELZN for this correction? In particular Mark fitted for
> >the atmosphere and the geometry at the same time.
> >
> >  But as far as we can tell DELZN does not fit for geometry. Is this
correct?
> >
> > Anyway, In the phase case the clocks are definitely zero, and so do
> >not need fitting for. The zenith depths calculated are a reasonable
> >4cm or so. The solutions have an odd curvature tho'. I attach the
> >plots of the values. Is this behaviour expected? I would have hoped
> >for a constant -- or nearly constant -- value for each site.  Further
> >more if one changes the order of the polynomial one would expect a
> >similar result -- I.e. the zeroth order would be the mean of the
> >higher order curves -- but this is not what we find.
> >
> >  When we tried reducing the order of the polynomial we got very
> >different solutions. What is _really_ being plotted as the routine
> >runs. When we limit the number of terms for the atmosphere we get
very
> >different ranges of values plotted (aparm(2)=1 is a few millimeters
> >(the attached zen_delay_0_order.ps), aparm(2)=2 is a some
millimeters,
> >aparm(2)=0 is a centimeters (the attached zen_delay.ps)). However at
> >all times the line goes through the data. How can this be so? Is the
> >data somehow different for each fit with different numbers of
> >polynomial terms.
> >
> >  The default solutions, however, do do a good job of correcting the
> >image quality: Reduced sidelobes, larger peak. However I worry that
> >this is very similar to what we would have got had one done a simple
> >selfcal. How is this different from a selfcal, other than smoothing
> >with a polynomial fit to the data?
> >
> >  Thanks for your help with these many questions,
> >
> >                     Richard
> >
> >----------------------------------------
> >Richard Dodson : Marie Curie Fellow
> >email : r.dodson at oan.es
> >Observatorio Astronomico Nacional
> >Apartado 112 E-28803 Alcala de Henares
> >Phone: (+34) 91-8855060/61, 62 (fax)
> >----------------------------------------
> >
> >