[daip] AIPS cannot see SN-tables

[Eric Greisen]

On 06/30/2015 03:33 AM, Franz Kirsten wrote:
> Hi Eric,
>
> in an attempt to speed up my data processing I am using ParselTongue to
> parallelise FRING. The way I do this is that I run multiple instances of
> FRING on the same data but each instance does a separate source and/or
> scan (eventually, I'd like to just merge all the resulting SN-tables).
> Things run smoothly and the SN-tables are being written out to disk. The
> trouble however is, that AIPS cannot see all of the new SN-tables but
> only the lowest one or two. The tables do exist though as I can see them
> in my data areas (e.g. SND04M00A.01G;). While running things in parallel
> ParselTongue seems to execute a clrstat-command at some point -- I fear
> that this actually prevents some info about the total number of existing
> SN-tables to be written to the header of the file. Can you think of a
> way to let AIPS know about these SN-tables? I tried to rebuild my
> catalogue with RUN RECAT, hoping it might pick up all the files related
> to the particular catalogue entry and user number but without success.
>
> Many thanks for any insights/help you could give me!
>
> Kind Regards,
> Franz

Your parallelism is biting you with a standard parallel problem.  AIPS 
records in the image header the maximum table version number.  If 10 
processes all grab the header at once, then all 10 will collide making 
SN table version 1.  You seem to have avoided that issue.  So, let's say 
that all 10 start in sequence and each makes SN version # equal the 
sequence number.  But, in less all 10 end also in sequence, the highest 
number shown in the header will be the one attached to the process that 
ends last.  If the CLRSTAT was not executed, then only the first of the 
10 processes would run and all others would be blocked by the first 
process's WRIT status.

I think you need a new verb that checks for existing extension files and 
sets the maximum number based on the files' existence.  The RECAT 
process (outside AIPS) uses the header files' information to rebuild the 
central catalog,but it does not do the search for extension files not 
known in the header.

Eric Greisen