[daip] AIPS timeout

[Eric Greisen]

Anita M. S. Richards wrote:
> Dear 'Daip',
> 
> We have a recurrent error with Dec08 AIPS, in that tasks time out
> AIPS 1: TASK HAS NOT BEGUN IN   15.1 SECONDS
> AIPS 1: Begin check for any 'standard' scratch files
> AIPS 1: Scratch files -- destroyed:    0  still active:    0
> 
> The task itself sometimes runs to completion but terminates any runfile or 
> script.  This makes it impossible to run the MERLIN archive pipeline so it 
> is quite a nuisance for a lot of people.  The error is sporadic but 
> frequent and is seen on a number of Linux workstations for a wide variety 
> of tasks and datasets and AIPS numbers and users.  We have a central 
> (binary) installation and Ant has recently updated it in an unsuccessful 
> effort to solve the problem.
> 
> There is no obvious cause, such as processor/memory overload, AIPS 
> confilcts or anything else.
> 
> We have Dec07 installed and working without this problem; however it is 
> the version prior to the patch for the CLCAL bug and we are reluctant to 
> upgrade it in case that also breaks everything!
> 
> The only help which we have managed to find suggests using compress, which 
> does not help.
> 
> Two questions:
> 
> It is possible that this is a local problem.  In that case, how can we 
> increase the timeout time, please? (I had a quick look at Going Aips but 
> could not find anything).
> 
> Please do you have any idea what is causing the problem and if so, whether 
> it is an AIPS bug or something which we are doing?

Sorry to be so slow in answering.  We have been racking our minds to 
think about circumstances when we see this message.  It is most common 
when we are debugging code and for some reason it does not even start - 
but that does not apply to you.  A suggestion was made - that when 
memory is crowded (both real RAM and swap) then tasks will try to start 
but not proceed properly until the memory becomes available.  31DEC08 
uses a lot more memory for the XAS TV than previous versions and 
pseudo-AP tasks like IMAGR now can take more memory than they used to 
take (although they take only what they need and only after the task has 
started).  Try looking at "top" to see how memory is used.  If your 
machines have < 1 Gbyte then I would expect problems in many cases.  At 
1 or 2 Gbyte, it would take heavier loading to cause problems (e.g. 
multiple TVs, emacs, browsers, mail readers, etc.)

I cannot think of much that differentiates 31DEC08 from 07 other than 
XAS and really heavy IMAGRs, CALIBs with models, et al.  (They do 
multiple facets/resolutions at once now and so use more memory.)

Eric Greisen