[daip] Questions

[Eric Greisen]

Vicas Dana wrote:
> Dear Eric,
> 
> It is the LITTLE THINGS Dana again.
> 
> I am very happy to have calibrated, combined data so we have started to 
> seriously chase for final cubes with the MS CLEAN implementation in 
> Aips. We like the results, although we are still fine tunning our 
> parameters. We are worried about cleaning too much at certain scales 
> especially since the Message window is repeatedly giving the following 
> warning (in red):
> 
> uhsclu> IMAGR1: ALGMEM:  1024 - 0 cells, with       79441 Pts
> 
> uhsclu> IMAGR1: Field    2 min =  -34.1 MilliJy,max =   34.3 MilliJy
> 
> uhsclu> IMAGR1: BGC Clean: using  103 cell beam + residuals >   583.06 
> MicroJy
> 
> uhsclu> IMAGR1:    19985 Residual map points loaded
> 
> uhsclu> IMAGR1: Field    2  FACTOR  0.0000  GAIN 0.10000
> 
> uhsclu> IMAGR1: SOMETHING IS GOING WRONG - ABANDON CLEAN
> 
> uhsclu> IMAGR1: Field    2 min algorithm flux= -766.337 MicroJy iter=    
> 37386
> 
> uhsclu> IMAGR1: Field    2 Clean flux density=    8.700 MilliJy    37386 
> comps
> 
> uhsclu> IMAGR1: Total Cleaned flux density     = -646.522 MilliJy   
> 980915 comps
> 
> uhsclu> IMAGR1: ALGMEM field    2 Ipol gridded model subtraction chns    
> 1-    1
> 
> uhsclu> IMAGR1: ALGMEM:  1024 - 0 cells, with       79441 Pts
> 
> uhsclu> IMAGR1: Doing no flagging this time
> 
> uhsclu> IMAGR1: Field    2 min =  -34.1 MilliJy,max =   34.3 MilliJy
> 
> uhsclu> IMAGR1: BGC Clean: using  103 cell beam + residuals >   571.58 
> MicroJy
> 
> uhsclu> IMAGR1:    19979 Residual map points loaded
> 
> uhsclu> IMAGR1: Field    2  FACTOR  0.0000  GAIN 0.10000
> 
> uhsclu> IMAGR1: Field    2 min algorithm flux= -575.896 MicroJy iter=    
> 39949
> 
> uhsclu> IMAGR1: Field    2 Clean flux density=    5.394 MilliJy    39949 
> comps
> 
> uhsclu> IMAGR1: Total Cleaned flux density     = -649.828 MilliJy   
> 983478 comps
> 
> uhsclu> IMAGR1: ALGMEM field    2 Ipol gridded model subtraction chns    
> 1-    1
> 
> uhsclu> IMAGR1: ALGMEM:  1024 - 0 cells, with       79441 Pts
> 
> uhsclu> IMAGR1: Field    2 min =  -34.2 MilliJy,max =   34.4 MilliJy
> 
> uhsclu> IMAGR1: BGC Clean: using  103 cell beam + residuals >   523.54 
> MicroJy
> 
> uhsclu> IMAGR1:    19985 Residual map points loaded
> 
> uhsclu> IMAGR1: Field    2  FACTOR  0.0000  GAIN 0.10000
> 
> uhsclu> IMAGR1: Field    2 min Clean flux    = -569.954 MicroJy iter=    
> 42586
> 
> uhsclu> IMAGR1: Field    2 Clean flux density=    3.849 MilliJy    42586 
> comps
> 
> uhsclu> IMAGR1: Total Cleaned flux density     = -651.374 MilliJy   
> 986115 comps
> 
> uhsclu> IMAGR1: ALGMEM field    2 Ipol gridded model subtraction chns    
> 1-    1
> 
> uhsclu> IMAGR1: ALGMEM:  1024 - 0 cells, with       79441 Pts
> 
> uhsclu> IMAGR1: Field    3 min =  -47.3 MilliJy,max =   47.8 MilliJy
> 
> uhsclu> IMAGR1: BGC Clean: using  103 cell beam + residuals >   525.28 
> MicroJy
> 
> uhsclu> IMAGR1:    19989 Residual map points loaded
> 
> ...
> 
> Could you give more details about how does this message come about?In 
> this particular case does it mean that we are overcleaning in Field 2?
> 
> Another issue we are tackling is regrading the very nearby dwarfs which 
> have been observed as mosaics in D array. I understand there is VTESS 
> which deals with mosaics in AIPS, using a Maximum Enthropy algorithm. I 
> was wondering if it is possible to use multi-scale clean algorithm. And 
> if so how would one go about doing that in AIPS?
> 
> Lastly, as we are dealing with huge amounts of data, I would like to 
> speed up the imaging time as much as possible. At the university I have 
> access to a cluster, which at the moment is only able to help me in
> speed terms, by having a bigger internal memory and more space on disk. 
> I was wondering if you could give any reference of some succesful 
> attempts of parallelizing the imaging process?

1. This message is produced when
          IF (IPOL .AND. (ABS(FLUXG(LFIELD)).LT.0.8*STRFLX(KFIELD))
      *      .AND. (CCMIN.GT.1.3*PCCMIN)) THEN
     which says that the abs value of the current total for the facet is
     < 0.8 the starting absolute value and the current component is 1.3
     times the starting value.  In other words clean is diverging.

     But the total flux over all facets is large and negative so I would 
suggest that you are going rather wrong in more serious ways.  Are you 
watching the Clean process on the TV.  You should be.

2. VTESS is a problematical program when 3D effects are significant 
although we may be about to solve that one.  I could not get it to do 
what I wanted with good S/N HI galaxy data with good prior images - its 
convergence criteria are suspect although some people with very good 
data get very good results from it.  The MEM should cover the 
multi-scale attributes.  We do not do mosaicing in the sense of 
simultaneous deconvolutions in AIPS.  However, if you reduce each 
pointing separately, VTESS will do the proper feathering to put them 
together.

3. So far as I know, there are no imaging programs that run parallel. 
We are looking at getting AIPS to farm out embarrassingly parallel 
problems to multiple nodes - e.g. different spectral channels to 
different nodes.  We will also investigate multi-threading in 
appropriate cases as well.  But neither will be available to you in a 
timely fashion.

Eric Greisen