[daip] help with "MAXPIXELS LOADED TO AP"
Tara Chaves
tchaves at astro.queensu.ca
Wed Jan 7 12:07:38 EST 2004
Hello wonderful people of the VLA.
I'm having a problem. My dataset (AL576) is ginormous... good for
astronomy, bad for computing. The HI-only (continuum-subtracted,
source-split) UV data file has over 4 million visibilities. There are 127
channels.
Four times, I tried to clean the data using IMAGR. Four times it crashed
with errors I have never seen before. Since you are all more experienced
than me with AIPS, perhaps you can see what I'm doing wrong. First, here
are the inputs I'm using:
AIPS 1: IMAGR: Wide field imaging/Clean task
AIPS 1: Adverbs Values Comments
AIPS 1: ----------------------------------------------------------------
AIPS 1: INNAME 'NGC 3556 ' Input UV data (name)
AIPS 1: INCLASS 'HIONLY' Input UV data (class)
AIPS 1: INSEQ 1 Input UV data (seq. #)
AIPS 1: INDISK 1 Input UV data disk drive #
AIPS 1: SOURCES 'NGC3556 ' Source name
AIPS 1: QUAL -1 Calibrator qualifier -1=>all
AIPS 1: CALCODE ' ' Calibrator code ' '=>all
AIPS 1: TIMERANG *all 0 Time range to use
AIPS 1: SELBAND -1 Bandwidth to select (kHz)
AIPS 1: SELFREQ -1 Frequency to select (MHz)
AIPS 1: FREQID 1 Freq. ID to select.
AIPS 1: SUBARRAY 0 Sub-array, 0=>all
AIPS 1: DOCALIB -1 If >0 calibrate data
AIPS 1: = 2 calibrate weights
AIPS 1: GAINUSE -1 CL (or SN) table to apply
AIPS 1: DOPOL -1 If >0.5 correct polarization.
AIPS 1: BLVER -1 BL table to apply.
AIPS 1: FLAGVER -1 Flag table version
AIPS 1: DOBAND -1 If >0.5 apply bandpass cal.
AIPS 1: Method used depends on value
AIPS 1: of DOBAND (see HELP file).
AIPS 1: BPVER -1 Bandpass table version
AIPS 1: SMOOTH *all 0 Smoothing function. See
AIPS 1: HELP SMOOTH for details.
AIPS 1: STOKES ' ' Stokes parameters (see HELP)
AIPS 1: BCHAN 0 Low freq. channel 0 for cont.
AIPS 1: ECHAN 0 Highest freq channel
AIPS 1: CHANNEL 0 Restart channel number
AIPS 1: NCHAV 1 Number of chan. to average.
AIPS 1: CHINC 1 Channel incr. between maps.
AIPS 1: BIF 0 First IF in average.
AIPS 1: EIF 0 Last IF in average.
AIPS 1: OUTNAME 'NGC 3556 ' Output image name (name)
AIPS 1: OUTDISK 1 Output image disk drive #
AIPS 1: OUTSEQ 0 Output seq. no.
AIPS 1: OUTVER 0 CC ver. no (Continuum only)
AIPS 1: *** SET OUTVER ON RESTARTS
AIPS 1: IN2NAME ' ' UV work file name
AIPS 1: IN2CLASS ' ' UV work file class
AIPS 1: IN2SEQ 0 UV work file seq
AIPS 1: *** SET TO KEEP WORK FILE
AIPS 1: IN2DISK 0 UV work file disk
AIPS 1: CELLSIZE 1.2 1.2 (X,Y) size of grid in asec
AIPS 1: IMSIZE 1024 1024 Minimum image size
AIPS 1: NFIELD 1 Number of fields (max 4096)
AIPS 1: DO3DIMAG -1 > 0 => use different tangent
AIPS 1: points for each field
AIPS 1: FLDSIZE *all 0 Clean size of each field.
AIPS 1: RASHIFT *all 0 RA shift per field (asec)
AIPS 1: DECSHIFT *all 0 DEC shift per field (asec)
AIPS 1: UVTAPER 0 0 (U,V) Gaussian taper
AIPS 1: units are kilo-lambda
AIPS 1: UVRANGE 0 0 Min & max baseline (klambda)
AIPS 1: GUARD 0 0 x,y guard band fractional
AIPS 1: radius
AIPS 1: ROTATE 0 Rotate image CCW from N by
AIPS 1: ROTATE degrees
AIPS 1: ZEROSP *all 0 0-spacing fluxes and weights
AIPS 1: SEE HELP!!
AIPS 1: UVWTFN 'NA' UV dist. weight function
AIPS 1: blank => uniform
AIPS 1: UVSIZE 0 0 Array size for doing uniform
AIPS 1: weights. 0 -> actual field
AIPS 1: size.
AIPS 1: ROBUST 0 Robustness power: -5 -> pure
AIPS 1: uniform weights, 5 => natural
AIPS 1: UVBOX 0 Additional rows and columns
AIPS 1: used in weighting.
AIPS 1: UVBXFN 1 Box function type when UVBOX
AIPS 1: > 0. 0 -> 1 round pill box.
AIPS 1: XTYPE 5 Conv. function type in x
AIPS 1: default spheroidal
AIPS 1: YTYPE 5 Conv. function type in y
AIPS 1: default spheroidal
AIPS 1: XPARM *all 0 Conv. function parms for x
AIPS 1: YPARM *all 0 Conv. function parms for y
AIPS 1: NITER 100000 Maximum # of Clean components
AIPS 1: BCOMP *all 0 Begin at BCOMP component
AIPS 1: Specify for each field.
AIPS 1: ALLOKAY 0 For restart: > 0 => beams
AIPS 1: okay, > 1 => work file too
AIPS 1: NBOXES 0 Number of boxes for Clean
AIPS 1: NB: field 1 only.
AIPS 1: CLBOX *all 0 Four coordinates for each box
AIPS 1: BOXFILE *all ' ' Input file of field params
AIPS 1: and Clean boxes; ' ' => use
AIPS 1: FLDSIZE, RASHIFT, DECSHIFT,
AIPS 1: NBOXES, CLBOX only.
AIPS 1: OBOXFILE *all ' ' Output file for final Clean
AIPS 1: boxes
AIPS 1: GAIN 0.2 Clean loop gain
AIPS 1: FLUX 1.000E-04 Minimum Clean component (Jy)
AIPS 1: MINPATCH 51 Min. BEAM half-width in AP.
AIPS 1: BMAJ 0 FWHM(asec) major axis Clean
AIPS 1: restoring beam.
AIPS 1: BMIN 0 FWHM(asec) minor axis Clean
AIPS 1: restoring beam.
AIPS 1: BPA 0 Clean beam position angle
AIPS 1: OVERLAP 0 1 => restore components to
AIPS 1: overlapped fields, >=2=>
AIPS 1: expect overlaps in Cleaning
AIPS 1: PHAT 0 Prussian hat height.
AIPS 1: FACTOR 0 Speedup factor see HELP
AIPS 1: CMETHOD ' ' Modeling method:
AIPS 1: 'DFT','GRID',' '
AIPS 1: IMAGRPRM *all 0 Task enrichment parameters
AIPS 1: (1) Antenna diameter (m)
AIPS 1: (2) Source Spectral index
AIPS 1: (3) Frequency scaling factor
AIPS 1: (4) > 0 -> SDI Clean factor
AIPS 1: (5) >0 => scale residuals
AIPS 1: (6) Half-width in x of box
AIPS 1: (7) Half-width in y of box
AIPS 1: (8) Filter components whose
AIPS 1: boxes
AIPS 1: GAIN 0.2 Clean loop gain
AIPS 1: FLUX 1.000E-04 Minimum Clean component (Jy)
AIPS 1: MINPATCH 51 Min. BEAM half-width in AP.
AIPS 1: BMAJ 0 FWHM(asec) major axis Clean
AIPS 1: restoring beam.
AIPS 1: BMIN 0 FWHM(asec) minor axis Clean
AIPS 1: restoring beam.
AIPS 1: BPA 0 Clean beam position angle
AIPS 1: OVERLAP 0 1 => restore components to
AIPS 1: overlapped fields, >=2=>
AIPS 1: expect overlaps in Cleaning
AIPS 1: PHAT 0 Prussian hat height.
AIPS 1: FACTOR 0 Speedup factor see HELP
AIPS 1: CMETHOD ' ' Modeling method:
AIPS 1: 'DFT','GRID',' '
AIPS 1: IMAGRPRM *all 0 Task enrichment parameters
AIPS 1: (1) Antenna diameter (m)
AIPS 1: (2) Source Spectral index
AIPS 1: (3) Frequency scaling factor
AIPS 1: (4) > 0 -> SDI Clean factor
AIPS 1: (5) >0 => scale residuals
AIPS 1: (6) Half-width in x of box
AIPS 1: (7) Half-width in y of box
AIPS 1: (8) Filter components whose
AIPS 1: neighborhood is weaker than
AIPS 1: IMAGRPRM(8) Jy. 0 -> don't
AIPS 1: (9) Radius in pixels for the
AIPS 1: IMAGRPRM(8) test.
AIPS 1: (10) multiplier of image size
AIPS 1: to get beam size: 0 => 2;
AIPS 1: 2, 1, 0.5 0.25 supported
AIPS 1: (11-16) Multi-resolution
AIPS 1: added controls
AIPS 1: (20) Retry factor (see help)
AIPS 1: (20) Retry factor (see help)
AIPS 1: NGAUSS 0 Number of resolutions to use
AIPS 1: WGAUSS *all 0 Resolutions in arc sec >= 0
AIPS 1: FGAUSS *all 0 Minimum flux for each resol.
AIPS 1: MAXPIXEL 0 Maximum pixels searched in
AIPS 1: each major cycle.
AIPS 1: DOTV -1 Display residuals on TV ?
AIPS 1: Start with field = DOTV
AIPS 1: BADDISK *all 0 Disks to avoid for scratch.
And here are the interesting error messages I've been getting...
1 8 08-NOV-2003 04:50:51 IMAGR WARNING: > MAXPIXELS
LOADED TO AP!!
1 4 08-NOV-2003 05:18:30 IMAGR Field 1 min = -1.1
MilliJy,max = 1.0 MilliJy
(This is the time it crashed. There was no error message recorded in the
AIPS MSSVR but I did get a "Oops:002" error in my system log. By the
way, I was also running all cleans in failsafe mode on my laptop.)
1 4 08-NOV-2003 15:30:13 IMAGR Beginning channel 2
through 2 with 1 IFs
1 2 08-NOV-2003 15:34:51 IMAGR OUVIMG: Sorting data to
make them fit
1 2 08-NOV-2003 15:41:46 IMAGR GRIDUV: Only 315 Rows in
1280k AP
1 2 08-NOV-2003 15:41:46 IMAGR GRIDUV: Max U Baseline
52430. lambda (= 626 cells)
1 2 08-NOV-2003 15:41:46 IMAGR GRIDUV: Frequency
1.415591E+09 Hz
1 8 08-NOV-2003 15:41:46 IMAGR GRIDUV: SORT ERROR: ROWS
340 348 OUTSIDE -202 112
1 7 08-NOV-2003 15:41:46 IMAGR OUVIMG: MAKING BEAM BEAM
image for CLEAN
1 8 08-NOV-2003 15:41:46 IMAGR OUVIMG: FROM UVdata work
object
1 8 08-NOV-2003 15:41:46 IMAGR CLNINI: ERROR OPENING
CLEAN process object
1 8 08-NOV-2003 15:41:46 IMAGR CLNUV : ERROR CLEANING
CLEAN process object
1 6 08-NOV-2003 15:41:46 IMAGR Deleting UV work file:
1 4 08-NOV-2003 15:41:46 IMAGR Destroyed 4 extension
files of type AN
1 4 08-NOV-2003 15:41:46 IMAGR Destroyed 1 extension
files of type FQ
1 4 08-NOV-2003 15:41:46 IMAGR Destroyed UV image file:
catno= 2 disk= 3
1 3 08-NOV-2003 15:41:46 IMAGR Purports to die of
UNNATURAL causes
1 5 08-NOV-2003 15:41:46 IMAGR avalon 31DEC02 TST:
Cpu= 892.8 Real= 3315
1 8 08-NOV-2003 19:27:33 IMAGR WARNING: > MAXPIXELS
LOADED TO AP!!
1 8 08-NOV-2003 23:07:14 IMAGR WARNING: > MAXPIXELS LOADED
TO AP!!
1 8 08-NOV-2003 23:26:32 IMAGR ALGSUB: ERROR: DATA
OUTSIDE OF UV GRID USED:
1 8 08-NOV-2003 23:26:32 IMAGR UMAX =
8.54268E+04 VMAX = 8.54268E+04 WAVELENGTH
1 8 08-NOV-2003 23:26:32 IMAGR OUMSUB: ERROR SUBTRACTING
CLEAN field number 0001
1 8 08-NOV-2003 23:26:32 IMAGR OUMSUB: FROM UVdata work
object
1 8 08-NOV-2003 23:26:32 IMAGR UVSUBM: ERROR SUBTRACTING
CLEAN field number 0001
1 8 08-NOV-2003 23:26:32 IMAGR UVSUBM: FROM UVdata work
object
1 8 08-NOV-2003 23:26:32 IMAGR CLNUV : ERROR CLEANING
CLEAN process object
1 6 08-NOV-2003 23:26:32 IMAGR Deleting UV work file:
1 4 08-NOV-2003 23:26:32 IMAGR Destroyed 4 extension
files of type AN
1 4 08-NOV-2003 23:26:32 IMAGR Destroyed 1 extension
files of type FQ
1 4 08-NOV-2003 23:26:32 IMAGR Destroyed UV image file:
catno= 2 disk= 3
1 3 08-NOV-2003 23:26:32 IMAGR Purports to die of
UNNATURAL causes
1 5 08-NOV-2003 23:26:32 IMAGR avalon 31DEC02 TST:
Cpu= 6665.8 Real= 20623
1 8 09-NOV-2003 17:37:16 IMAGR WARNING: > MAXPIXELS
LOADED TO AP!!
1 5 09-NOV-2003 18:16:26 IMAGR SOMETHING IS GOING WRONG -
ABANDON CLEAN
1 5 09-NOV-2003 18:17:32 IMAGR SOMETHING IS GOING WRONG -
ABANDON CLEAN
1 4 09-NOV-2003 21:40:40 IMAGR Beginning channel 16
through 16 with 1 IFs
1 2 09-NOV-2003 21:45:18 IMAGR OUVIMG: Sorting data to
make them fit
1 2 09-NOV-2003 21:52:19 IMAGR GRIDUV: Only 315 Rows in
1280k AP
1 2 09-NOV-2003 21:52:19 IMAGR GRIDUV: Max U Baseline
52442. lambda (= 626 cells)
avalon AIPS (31DEC02) 221 10-NOV-2003
12:34:42 Page 211
Pops Prior Date Time Task Messages for user 221
1 2 09-NOV-2003 21:52:19 IMAGR GRIDUV: Frequency
1.415933E+09 Hz
1 8 09-NOV-2003 21:52:26 IMAGR GRIDUV: SORT ERROR: ROWS
38 46 OUTSIDE -310 4
1 7 09-NOV-2003 21:52:26 IMAGR OUVIMG: MAKING BEAM BEAM
image for CLEAN
1 8 09-NOV-2003 21:52:26 IMAGR OUVIMG: FROM UVdata work
object
1 8 09-NOV-2003 21:52:26 IMAGR CLNINI: ERROR OPENING
CLEAN process object
1 8 09-NOV-2003 21:52:26 IMAGR CLNUV : ERROR CLEANING
CLEAN process object
1 6 09-NOV-2003 21:52:26 IMAGR Deleting UV work file:
1 4 09-NOV-2003 21:52:26 IMAGR Destroyed 4 extension
files of type AN
1 4 09-NOV-2003 21:52:26 IMAGR Destroyed 1 extension
files of type FQ
1 4 09-NOV-2003 21:52:26 IMAGR Destroyed UV image file:
catno= 1 disk= 3
1 3 09-NOV-2003 21:52:26 IMAGR Purports to die of
UNNATURAL causes
1 5 09-NOV-2003 21:52:26 IMAGR avalon 31DEC02 TST:
Cpu= 12051.8 Real= 37080
Now, a collaborator of mine suggested something about an archaic AP memory
thing filling up with the Fourier transforms and that I try using IMAGR
first to do the transforms and then use APCLN to do the cleaning. I did
that, used IMAGR to make a dirty cube, then APCLN with the same parameters
as above to clean the cube, only these data came out with a nice ring
around the galaxy. Most frustrating. For a given channel with HI
emission, cleaning with the above parameters using IMAGR results in a nice
image with no ring, but if I take the same channel and make a dirty imagr
with IMAGR first, then clean with APCLN, I get the ring. I have no idea
why.
I next tried cleaning three successive channels with HI emission, each
separately with IMAGR. I then tried using MCUBE to combine these three
2-D images into a cube. Here are the inputs I used:
AIPS 1: MCUBE: Task to collect a set of n-dim maps into a (n+1)-dim map
AIPS 1: Adverbs Values Comments
AIPS 1: ----------------------------------------------------------------
AIPS 1: USERID 0 User ID. 0 => current user,
AIPS 1: 32000 => any user.
AIPS 1: INNAME 'TEST ' Input name(name).
AIPS 1: INCLASS 'ICL001' Input name(class).
AIPS 1: INSEQ 1 Input name(seq. #). 0=>high
AIPS 1: First sequence # in the set
AIPS 1: INDISK 3 Input disk drive #. 0=>any
AIPS 1: IN2SEQ 2 Last sequence # in set.
AIPS 1: IN3SEQ 3 Sequence # increment.
AIPS 1: OUTNAME 'TEST ' Output name(name).
AIPS 1: OUTCLASS 'ICL001' Output name(class).
AIPS 1: OUTSEQ 4 Output name(seq. #).
AIPS 1: 0 => highest unique
AIPS 1: OUTDISK 3 Output image disk drive #
AIPS 1: 0 => highest with space
AIPS 1: DOALIGN 1 Alignment control parm
AIPS 1: AXREF 1 n+1 axis pixel of map INSEQ.
AIPS 1: 0 => 1
AIPS 1: AX2REF 2 n+1 axis pixel of map IN2SEQ
AIPS 1: 0 => opposite end from INSEQ
AIPS 1: NPOINTS 3 Number of pixels on axis n+1
>go
Here are the resultant messages:
AVALON> MCUBE1: Task MCUBE (release of 31DEC02) begins
AVALON> MCUBE1: Create TEST .ICL001. 4 (MA) on disk 3 cno 1
AVALON> MCUBE1: Writing sequence # 1 to plane 1
AVALON> MCUBE1: Blanking empty output plane 2
AVALON> MCUBE1: Blanking empty output plane 3
AVALON> MCUBE1: Appears to have ended successfully
AVALON> MCUBE1: avalon 31DEC02 TST: Cpu= 0.2 Real= 1
The resultant cube consists of one channel of data and then two
completely blanked channels. Yet the input channels do contain data.
I would be eternally grateful for any help or advice here on how to clean
this gigantic data set. Obviously, the UV data are much too big to ftp
down there but I am currently residing in Albuquerque and I'd be more than
happy to drive my laptop down to Socorro for someone to play with. My
schedule is open, any time is fine... besides, I still owe Greg Taylor a
beer!
Thanks so much for your help, O wiser-than-I AIPS-gurus!
Tara Mowery
PhD Candidate, Physics
Queen's University at Kingston
More information about the Daip
mailing list