[asac] Reply to ALMA Sciene Software Requirements Committee

[Karl Menten]

Dear ASAC members,

Attached you find a draft of the ASAC response to the ALMA 
Sciene Software Requirements Committee questions.
Please email me final comments by Friday, June 30.

Cheers,
	Karl

P.S. The document can also be found on:
http://www.mpifr-bonn.mpg.de/staff/kmenten/ssrc-response.txt

-----------------------------------------------------------------------------
Dr. Karl M. Menten   (kmenten at mpifr-bonn.mpg.de)
Max-Planck-Institut fuer Radioastronomie
Auf dem Huegel 69, D-53121 Bonn, Germany
Tel.: +49 (0)228-525297 * Fax: +49 (0)228-525435

-------------- next part --------------
MEMORANDUM

Date:  29-June-2000
To:    Robert Lucas, ALMA Science Software Requirements Committee
From:  ALMA Scientific Advisory Committee (ASAC)
Topic: Comments on issues raised by ALMA Science Software Requirements 
       Committee (Email message by Robert Lucas, 18-May-2000)  
------------------------------------------------------------------------
>
> to ALMA Project Scientists and Alma Science Advisory Committee Members:
> -----------------------------------------------------------------------
>
> Object: Project-wide issues of specific interest to Science Software
> Requirements
>
>   During our past six months of work in the Science Software
>   Requirements Committee we have encountered several issues on which
>   we agree that further input on your side will be needed before our
>   work can be completed. Our preliminary report (ALMA memo 293) has
>   been recently officially reviewed and we take this opportunity to
>   raise the following questions. These are mainly issues related to the
>   way ALMA is operated as a whole. For each issue we would like to
>   have either a definite answer or a baseline/fall-back
>   alternative. The questions are given in order of decreasing importance.
>
> 1) Array Scheduling
>
>   We have been assuming that ALMA will be dynamically scheduled so
>   that each project is observed during weather conditions (seeing,
>   opacity, wind) that allow its objectives to be achieved. This
>   implies dynamic scheduling in near real time (Dynamic Scheduling) as
>   stressed in mma memo 164, and in ALMA Project Book, Chapter
>   2(III.4). This has implications on the feedback from the PI, on the
>   basis of calibration data and images produced by the pipeline: this
>   feedback can only be offered at the end of one transit (observing
>   session), assuming the project observations are divided into several
>   transits either due to schedule optimization (as high elevations are
>   preferred) or intentionally by the insertion of breakpoints. This
>   limitation on `interactivity' has to be realized. We believe it is a
>   small price to pay for the increase in productivity expected from
>   dynamic scheduling.
>
>   An alternate scheduling method is the traditional interactive
>   observing, which should be available since it can be useful for test
>   purposes and for special -- timely and unforeseen -- observations.
>
> Our question: Is dynamic scheduling to be the default mode of
>   scheduling, accepting the restricted interactivity implied by this
>   mode ?
>

The ASAC very strongly recommends that dynamic scheduling should be the 
default mode.


>
> 2) Purpose of the pipeline
>
>   The data flow ALMA Project Book, Chapter 2(III.5) will include a
>   pipeline that may be used to  fulfill several purposes:
>
>   a) check that the data obtained can be calibrated, with feed back to
>     the observing process to use the optimal phase calibration cycle.
>
>   b) give feed back to the dynamic scheduler by monitoring the observed
>     phase noise.
>
>   c) produce calibrated uv data and maps of test quasars to check in
>     real-time the quality of observations, with feed back to the
>     dynamic scheduler.
>
>   d) produce calibrated uv data and maps of the project source(s) to
>     enable the PI to evaluate her/his data at the end of the observing
>     session and to proceed with scientific interpretation when the
>     project is finished (after improved reduction if needed), while
>     incrementing the ALMA data archives.
>
>   e) use calibrated uv data and maps of the project source(s) in order
>     to derive simple quality parameters (noise level, signal to noise
>     ratio) that may be used to define when the project goals are
>     attained.
>
>   f) use calibrated uv data and maps of the project source(s) in order
>     to derive similar or more sophisticated parameters (source size,
>     number of sources ... see examples in memo 293) to be fed back
>     into the project's observing process, which may then take
>     automatic decisions.
>
>   While we believe that a, b, c and d should be implemented as
>   baseline features, there is some concern that f) and maybe e) might
>   be too ambitious goals (software has a cost too) or even may
>   increase the distance between the astronomers and the instrument to
>   an unwanted level.
>
> Our question: Which degree of sophistication should be set as a goal
> for the ALMA data pipeline ?
>

a)-d): yes! In particular, d) means that calibrated images are the final 
data products! - This would result in a much lower useage threshold for
non-radioastronomers.

e) would be useful in certain cases, but without a source model in
general pretty difficult. 

f) This should be left to the astronomer.  To allow this and at the
same time not to slow down project completion too much, a project
could have predefined breakpoints, at which the astronomer evaluates
the data taken so far, and the subsequent course of the project
depends on the astronomer's decision.


>
> 3) Operational aspects
>
>   For the specification of GUI (graphical User Interfaces) a clear
>   description of the relative duties of the operators and local staff
>   astronomers will be needed, in view of the following tasks
>
>   - Allocation of antennas to simultaneous projects (e.g. a sub-array
>     for calibration, a sub-array used for astronomy, some antennas in
>     maintenance) ?
>

Operator, after consultation with local staff astronomer.


>
>   - Control on the dynamic scheduling process
>

Operator, but only in case of emergency.


>
>   - Communication with the PIs if needed
>

Local staff astronomer.


>
> Our question: Can the relative duties of the staff in charge of
> controlling the array be already outlined ?
>

See above


>
>
> 4) Policy on data propriety:
>
>   We are assuming, as is the usual practice in most observatories,
>   that only the proposing team has access to the data during a limited
>   proprietary period and that the data ultimately become public. The
>   actual length of the propriety period can be probably be set later,
>   but some policy questions may affect the software requirements, such
>   as:
>
>   4.1 is only the scientific data covered, or all header information
>   including monitoring data, or some header information only
>   (like coordinates and frequencies) ?
>

All the header information should be public immediately to avoid duplication.

For the data, a proprietary period of 1 year seems reasonable. This
applies to the target source data. Phase and flux calibrator data
should be public immediately.  

In certain cases, in particular for PhD projects, the proprietary period
might be extended.

For complex projects, such as surveys or projects requiring many 
configurations, it mmight be appropriate to let the proprietary period
start after all of the data have been collected.


>
>   4.2  if public data is reprocessed in the pipeline by others, to search
>   for unforeseen scientific results, does this start a new proprietary
>   period ?
>

We don't think that this would be appropriate - it would be easy to block data
forever by periodically reprocessing them.


>
> Our question: Can the proprietary data policy of ALMA be already
> outlined ?
>
> 5) Special modes
>
>   Some specific observations (Sun, Pulsars, ...) may need very short
>   integration times, fast frequency changes, ... for which the exact
>   scientific software (and hardware ?) requirements need to be
>   investigated in detail. For these issues we would like having
>   specialized astronomers as correspondents, so that we may include
>   their contributions in our requirements, in a coherent manner.
>

Arnold Benz will organize a discussion at the IAU General Assembly in 
Manchester (August 2000) on solar, stellar and possibly pulsar science 
requirements for ALMA. He will in addition also contact some other 
specialists in the field that will not be in Manchester.                  


> 6) Other details
>
>   - is an audio channel from the antenna cabins to the operator
>     planned ? (see comment 95)
>

Would be useful, but is trivial.


>
>   - We assume, as DSB mode is clearly an option for some frequency
>     bands, that side band separation has to be handled by software for
>     interferometric work in those bands.
>
> Robert Lucas,
>
> on behalf of the Science Software Requirements Committee