[asac] proposal review comments

[Rob Ivison]

> I would like to ask ASAC members to send your opinions/comments on 
> the draft to this mailing list preferably before the next telecon 
> especially for those who will not be able to participate it, and 
> then we can discuss this issue at the telecon based on the 
> information.

all -

here are some thoughts on the ALMA Proposal Review Process draft. i've 
listed them in the order that i encountered the various issues in the 
document, rather than by importance.

science categories: these appear sensible, both in number and scope, 
though i couldn't help wondering which box high-redshift radio 
galaxies might fall in? presumably "Cosmology and the high-redshift 
Universe" rather than "Galaxies and galactic nuclei". a definition of 
"high redshift" is all that is required (z>0.1?)

review panels: i can see that panels consisting of 9 members is 
appealing from the point of view of regional representation. however, 
in my experience members of panels comprising >6 people tend to assume 
that other folk will do the hard work and find the important little 
flaws. i would advise more, smaller panels (which is, apparently, 
contrary to previous ASAC advice).

125 proposals per reviewer seems, to me, absurdly high.

i like the idea that panel members serve 3 seasons. the biggest 
complaint about HST TACs is that the churn rate is too high. the 
biggest complaint about UK TACs is that the churn rate is too low 
(Chairs are often on panels for 6 years). i think the numbers here are 
about right.

grading system: seems sensible, with lessons learnt from previous 
weather-sensitive facilities.

review process: why two secondary assessors? reducing this to one 
secondary assessor would be an easy way to significantly reduce the 
overall burden on the assessors and thereby increase the quality of 
the assessing.

it would help to write down explicitly whether a proposal that is 
"borderline technically infeasible" can or cannot be "re-written" to 
some degree by the panel. i don't care either way, so long as the rule 
applies to everyone.

i would recommend that the science grades for each assessor are 
normalised to a common mean and stdev, and that assessors are allowed 
to give a null ranking if they feel it is appropriate for whatever 
reason.

i didn't get a clear view from the document about how the 
recommendations of the different review panels will be merged. we 
should not ring-fence time for themes, so allocations based on the 
number of panels or the over-subscription rate are to be avoided. i 
would suggest that the APRC must explore a certain number of proposals 
near the A/B, B/C, C/D borderlines of each panel and form a conclusion 
re: their relative merits.

this appears to be weakest part of the document. it ducks for cover 
when faced with the thorny issue of how to avoid ring-fencing time for 
themes.

the issue of how time (and good weather) is to be shared accurately 
between regions is handled simply and straightforwardly.

the formulae for sharing time amongst co-Is appears to be sensible.

i am not happy with the 5% cap on open skies, nor about the idea that 
NA should pick up the bill if this is exceeded. i do not believe that 
a cap should be discussed in the document, or that one is justified. 
however, i am prepared to accept that no-one will give a stuff about 
what i think on this issue ;-)

cheers,

Rob