[mmaimcal]Re: WVR treatement in System Design Description

[Larry D'Addario]

Richard,

Thanks for your well-considered comments.  I understand your concern
that the WVR performance is not being treated properly or fairly, and
I'll do my best to address that.

In summary, I'm trying to produce reasonable estimates of the
*corrected* atmospheric effects, including what can realistically be
expected from the WVRs.  The job that I've been assigned is to set the
technical requirments on the instrumental stability, and I believe that
there is no point in making these much tighter than what nature imposes
on us (after all available corrections).  I'm using my best judgment,
with all data available to me, to decide what's realistic.  Apparently
you disagree.  I am happy to be corrected by those who understand the
situation better than I, but it would help to have some objective
evidence that a given level of performance is achievable.  We need
experimental demonstrations, or at least *convincing* simulations. 
Perhaps there is data that I have not seen.  In the absence of this, I
still think it is rather *optimistic* to assume that ALMA will
*routinely* achieve the lowest residuals that have ever been
demonstrated.

This job of estimating the corrected atmospheric fluctuations is rather
different from setting the specifications on an instrument.  In the
latter case, we are talking about things that can be measured in the
lab.  We can anticipate improvements in the state of the art that will
result in better performance.  You say that I've been willing to do this
for the signal path of ALMA, and that it is inconsistent not to do the
same for the WVRs.  Actually, I am quite willing to accept your
predictions of the performance of these devices as *instruments* -- that
is, of their measurable parameters like noise temperature and
stability.  But it is a long way from there to the mm/submm delay
fluctuations.  All that gets measured directly is the brightness
temperature of the sky at several frequencies, and this is subject to
both systematic and random errors.  From these measurements one attempts
to derive the column density of water vapor, and this is requires
assumptions about atmospheric physics and geometry which are also
subject to error.  Given the water vapor, one then attempts to estimate
the mm/submm delay, and this process contains still more uncertainties
(the biggest of which, I suppose, is that the dry air fluctuations
cannot be estimated).  If you have evidence that this final result can
be substantially better than anybody's experience, please tell us about
it.

Note that more explanation about the phase stability requirements is
given in another (new, draft only) document, "System Level Technical
Requirements," available on ALMAEDM under /IPTs/Systems/Documentation as
doc #80.04.00.00-004-A.


Now let me answer some of your specific points.

Richard Hills wrote:
> In the second full paragraph of page 12 (part of section 3.1.1.2) it
> states that the WVR correction goal under 5th percentile conditions at
> 45 degrees elevation is 68 fs.  (This is fact the value derived from the
> present WVR *specification*, not the *goal*.  Our goals, and the
> presently predicted performance, correspond to a considerably lower
> figure.)

I was unaware of any tighter goals than the stated specifications.  Is
this documented somewhere?  Regardless of goals or specs, what can we
realistically expect, and how is that expectation justified?

> The paragraph then goes on to disregard this figure on the
> grounds that it is unrealistic to expect ALMA to do better than has
> already been demonstrated on existing telescopes.  Instead the figure of
> 83 fs is taken, and this is then adopted as the specification for the
> "instrument" - telescopes plus electronics.  I am not aware of this sort
> of defeatist policy being adopted anywhere else in the project.  I think
> that it is inappropriate and that this whole section should simply be
> removed.

I can't just remove it because I need to assume *something* about the
corrected atmosphere in order to set the instrument's requirements.  See
above for a discussion of the difference between the specificatons of
some hardware and the expected residuals of a very indirect measuring
process.

> ...  I feel, however, that putting the atmosphere, the
> structure and the electronics together is not helpful because they
> behave in such different ways with respect to conditions.  As you
> explain, the atmospheric term is highly dependent on the state of the
> atmosphere, the baseline and the techniques used to correct for it.

That's true, so I'm not putting them together, although perhaps the
layout of my Table 4B makes it look that way.  Since the atmosphere is
highly variable, it's treated separately.  As explained in the last
paragraph on p. 11, we first attempt to determine what the residual
atmosphere will be like at the 5th percentile.  Then we ask that the
instrument be at least as good as this.  We do not start with a
pre-determined total and attempt to allocated it between atmosphere and
instrument.

> In  table 4B you have focussed on the numbers for very good atmospheric
> conditions.  The figures given for the telescope structure are, by
> contrast, close to the maximum allowed.  On these short timescales,
> these residuals are likely to be dominated by winds, and the figure of
> 50 fs  (15 microns), which is still the one in given in the latest
> version of the antenna specification that I have seen, is required to be
> met for winds of 6m/s in the daytime or 9m/s at night (with an
> appropriate spectrum of gusts).  It was not clear to me whether you were
> proposing that this 50fs figure be replaced by your new one of 42 fs
> and, if so, for what environmental conditions it would apply.  In any
> case the point is that the structural contribution to the error should
> be less than this value for much of the time.  (I have just noticed that
> in this document you define the coherence error as being the deviation
> within a 1 second time interval.  The errors on that short a time should
> indeed be very small for the antennas, so 42fs seems much too large an
> allocation.)

These are good points.  The allocation between structure and electronics
is somewhat arbitrary (at 25%,75% respectively) and could be
re-considered.  Please remember that the numbers in the 2003-Dec-15
version of my document are still preliminary.  The resulting 42 fsec
number for structural fast fluctuations may be pessimistic, but reducing
it would not loosen the electronics requirement very much.  The more
important number is the long term (300-1000 s) variation, which is held
at 14 fsec, far below the 50 fsec in the official antenna
specification.  I consider this reasonable in view of the fact that the
official spec is contractural and must be *guaranteed* by the
manufacturer, whereas here we want to an estimate of what will be
achievable.

You are right that the contribution of the structure will be
wind-dependent, and this has not been explicitly considered.  I do not
know, for example, whether good submm transparancy and low delay
fluctuations are usually accompanied by low wind or high wind.  If good
sky usually goes with high wind, we may have less submm weather than we
would like.

> 
> This whole section, 3.1.1, is in fact dealing with the specification of
> the LO.  By constrast to the atmospheric and antenna contributions, I
> think it is likely that the loss due to lack of coherence in the LO
> signals will be largely independent of conditons,  i.e. it will be there
> all the time.  It seems to me much simpler therefore just to look at the
> actual value of the loss as a function of frequency and decide whether
> or not this is acceptable.  For the proposed value of 72fs, for all the
> electronics, I get that the efficiencies are:   83.1% at 950GHz,  91.5%
> at 660GHz and 97.6% at 345GHz.  These seem to me to be pretty reasonable
> figures.  One can see that one is getting into diminishing returns here
> - reducing this error by root 2 to 51fs would only improve the signal to
> noise by 9.7% at 950GHz, and 1.2% at 345GHz.

Here you are reverse-analyzing the results.  The instrument's
requirements were set in the way that I've explained:  instrument better
than atmosphere 95% of the time.  Having done that, you can ask what the
statisitical distrubion of efficiency will be.  This depends on many
things, including the allocation of observing time by wavelength.  One
could also take the opposite approach:  let the instrument builders say
what they will achieve, and then see what efficiency we get.  I hope you
can see that this is a chicken-and-egg problem.  Anyway, I do not think
it's useful to calculate what the electronics alone will do (as if the
atmosphere were absent and the structure were perfect).

...
> I have one final point about the electronics coherence number:  it is
> given in terms of a delay, and we generally assume that this is an error
> at the RF frequency so that it causes a loss which scales with RF
> frequency squared.  There will however be some contributions for which
> this is not true - e.g. phase noise on the second LO.

True, but this is a level of detail beyond the scope of the document.  I
am trying to provide a single-parameter requirement, so it involves some
simplifications.  It is implied, and it is reasonable to expect, that
the dominant cause of phase fluctuations will be the first LO, and most
components of the first LO produce phase changes proportional to LO
frequency, so characterizing them in time units is convenient. 
Fluctuations in the RF signal path are also likely to be proportional to
frequency.  If other subsystems (including the 2nd and 3rd LOs and the
IF/baseband signal path) have significant contributions, then this needs
to be addressed in lower-level specifications.


> Turning now to the accuracy figures, it is interesting to see that these
> have come down from 50fs for each of the structure and the electronics
> to 14 and 24fs.  As I understand it this is a result of taking the
> figure of 28fs from the simulations of the fast-switching and
> distributing it in the same way as the coherence.

When you say "come down" I do not know what you are comparing against. 
Please take the current numbers with a grain of salt; while I think that
they are roughly correct, they are subject to refinement.  All
instrumental numbers are now considerably *larger* than in the old
Project Book, which is the only thing we had prior to this.

> Again, this section
> on the LO spec does not seem to be place for setting the requirement on
> the antenna, but I note that the figure of 14fs seems very small
> compared with the 50fs in the current antenna spec.

To set the LO spec, *something* had to be assumed about everything else,
including the structure.  I could have assumed that the structure is
perfect, thus assigning all the error to electronics, but I chose not to
do that.

> I note that the
> 50fs already assumes that most terms are subtracted out by going to a
> reference source - the spec refers to a 2 degree distance on the sky and
> timescales of up to 3 minutes.  At the very least, it seems to me that
> we have some problems with consistency here.  (I find the antenna spec
> hard to follow on this point - does anyone know if it has it gone out
> with that section in the form it was in that Al circulated in early
> December?)

As discussed above, the numbers I've given have nothing to do with the
antenna spec.  I included the 50 fsec number in Table 4B for comparison,
but perhaps I should delete it.

> 
> Focussing on the electronics number, I see that you have taken the
> timescale for the accuracy requirement to be from 1 second to 1000
> seconds.  I assume that this is because it is assumed that 1000s is the
> maximum time between observations of bright broad-band point sources to
> tie together the phase of the ~100GHz system used for the fast switching
> and that of the band being used for the astronomy.  Is that right?   If
> so, then shouldn't we really separate the contributions that are common
> between the two systems from those that are not?   The common errors on
> timescales longer than the fast-swtiching cycle are presumably not
> important, where those that are independent between frequency bands will
> contribute on both the source and the calibrator.  This last point
> suggests that the imdependent errors need to be a factor of root 2 lower
> if the intention is to limit the errors on the final data to these values.

You are quite right that, on the one hand, only the differential errors
matter; and on the other hand the differential error is the sum of the
errors from each of the bands.  I've thought about this quite a bit.  We
really don't know enough about how to separate differential from common
mode errors, so I've arbitrarily decided to interpret the requirement as
applying to *all* of the fluctuation from *each* band separately.  That
is, I'm assuming that these two things roughly cancel.  When we
understand things better, this can perhaps be refined.


> 
> Moving on to section 3.3.4, page 19, para 4, which again concerns the WVR.
> 
> 1) The contributions due to noise in the radiometer and the scaling
> error are independent so the requirements, should be given separately as
> 0.02deltaL and 0.01w + 10microns. These should be added in quadrature.
>  This also needs to be fixed in the appendix.  This was my fault for
> writing the expression down incorrectly on a slide shown at ALMA week
> last year, although I thought I sent round an e-mail correction.  In the
> examples,  for deltaL = 500microns and w = 1.8mm the (rounded) value
> should be 30microns and for 100 and 1.6 it is 16.

This is reasonable, but it should be documented (and email does not
count!).  In fact, about the only documentation we have is the
powerpoint file from ALMA week, and this is not very satisfactory. 
Nevertheless, I'll try to fix up this point in the next edition, at
which time I hope that I can cite a proper reference.

> 
> I think the comments starting "but there is no assurance that this can
> be achieved..." up to "...more precise" should be removed:  there is no
> equivalent discussion on other subjects.  If you want to put in a
> warning, then it is worth pointing out that fluctuations in the dry
> component obviously cannot be corrected in this way and that performance
> is likely to deterioratate in the presence of clouds.

Are not these things -- dry air and clouds -- already included in the
specification (mainly the 2% scaling error)?  If not, then we need
another error term to account for them.

> As regards the
> next sentence, it was certainly my understanding that the 2%
> proportional error includes the errors in atmospheric modelling and also
> that it applies to changes in the total water vapour path due to e.g.
> switching to a reference source, if one wanted to use it in that way.

It was my understanding that the 2% error applies at constant air mass,
since the scaling factor can vary with air mass.  Thus, if you want to
measure the difference in water vapor for two sources at different
elevations, then you can expect the error to be greater than 2%.  I also
understood that the spec applies only to the *fluctuations* about the
average value in an interval of ~5 min, so the average value as an
*absolute* delay or column density of water vapor could be in error by
far more than 2%.  If I misunderstood any of this, then I'm ready to be
corrected.

> 
> Turning to different points:
> 
> Section 2.3 para 4.  The point is made that the attenuators should not
> be changed over a phase calibration cycle.  In practice this presumably
> means a cycle beginning and ending on the source which ties together the
> phase of the astronomical receiver and the 100GHz Rx.  This suggests to
> me that the software would need to be monitoring the IF level e.g. as a
> source was setting and the atmospheric noise rising, and deciding
> whether a change in attenuator was needed  The sequence would then be:
> go to the reference source and check the phase;  then switch the
> attenuator and measure the phase again;  then go back to the source.

This sequence, where we use an astronomical method to calibrate the
attenuator's phase variation, is not what I have in mind.  It might be
applied as a last resort, if we get in trouble.  Instead, the
attenuators should just be left fixed for long periods of time.  In
practice, I expect that we will have enough dynamic range that
attenuator changes are not needed except when some other major change of
setup occurs (e.g., change of band or a large frequency change within a
band) or a substantial change in weather conditions occurs.

>  Are the software people aware of this - i.e. is it in the software
> requirements?  A similar point presumably applies to adjustment of the
> LO power.

Sure.  Whenever any setup change occurs (where attenuator setting and LO
power are just two of many examples), the instrumental phase should be
considered unknown until it is calibrated.

> 
> Section 3.3.3.  You say that no special instrumentation is "planned" for
> measuring atmospheric extinction.  In fact a good deal of work has gone
> into discussing what equipment would be helpful (and indeed required) in
> meeting the amplitude calibration accuracy requirements and a set of
> outline specifications for these ahve been drawn up.  It is an
> interesting question as to whether any budget for these has been
> assigned.  More generally it seems to me that more work is needed in
> relating the technical specs you are giving here to the scientific
> requirements, especially on calibration.

The purpose of my document is to describe what we are planning to
build.  "Discussing what equipment would be helpful" is a far cry from
actually planning to build it.  There is no such plan or budget.  When a
comprehensive calibration scheme is available, perhaps it will depend on
the availability of extinction monitoring equipment.  Then maybe such
equipment will get into the construction plan and budget.

> 
> Finally some pedantic corrections of typos, etc:
> 
> Page 3.  Band three is given as 84-116GHz in the table and 86-116GHz in
> the text immediately below.

New vs. old numbers.  Sorry, will fix.

> 
> Page 8. the section labelled 6.1.3.4 should be 2.6.1.4 ?

Right, will fix.

> 
> Page 8.  Last paragraph but one.  I thought that given that we have a
> limited number of bits, there is an improvement in signal to noise from
> over-sampling.  If so the statement that there is no loss of sensitivity
> is not strictly correct.

Perhaps my wording is not clear.  Here I am discussing a feature inside
the cross-correlator.  If the *sampling* rate is twice Nyquist, then the
sensitivity (in any one spectral channel) is unchanged if we compute
lags that are two samples apart, compared with computing all the lags
spaced by one sample.  The fact that oversampling produces a slight
increase in SNR compared with Nyquist sampling was not the subject of
this sentence.

> 
> Top of page 9.  K is introduced without, I think, any definition.

This is a relic of an earlier version, and needs to be re-written. 
Thanks for pointing it out.

> Lower on page 9 - 4th bullet - "filteR bank"

OK.

> Page 11. Table 4A  Band 3  range given as 96-204GHz  - should be 104 ?

Right.

> That's all I have at the moment.

Thanks for giving it such a careful read.  It's a big help.

Regards,
Larry