[daip] Fwd: Use of EDITR in AL730 project
Barney Rickett
bjrickett at ucsd.edu
Mon Feb 23 20:24:09 EST 2009
Dear DAIP & Amy (not sure to whom I should send this plea for help)
I have followed the suggestion by Amy - but still cannot get editr to
add flags for my uv data... as I described in my original email
below.
I used extd and tacop to copy my highest FG table to FG1 and then
used uvcop to create a new catalogued file.
I then used edita to flag the many 'ty' values that were above 300K
in the copied file. On rerunning edita the high ty are not shown -
evidently edita worked OK. I then tried editr, uvplt and wiper on
data from one secondary calibrator source (of our multi-source data)
in various ways.
I found that, even though editr seems to exit successfully and msgsrv
states that rows were added to my FG table, the points I had flagged
keep coming back in both editr and uvplt.
However, when I used 'wiper' I was able to flag points and they did
not reappear in uvplt or editr. I conclude that wiper is working
for me but editr is not.
I then restarted editr on the data with the FG version generated by
wiper. I flagged another clear set of bad points and saw that FG had
gone from 5 to 6, but these points are still present in uvplt with
flagver = 6 (and also in a re-run of editr).
I am attaching a text file with the inp to editr, the state of
headers from imh and the output on msgsrv during the use of editr.
I aplogise for bothering you all with my difficulties but I am
stymied at present. After very many hours trying to edit the 5 days
of data from the VLA, I cannot make the flagging work with EDITR. (I
see that the number of rows in the FG table is now only 179. I
removed all data from one antenna and see several messages "EDTINC:
TIC MARK ALGORITHM FAILED! CONTINUING." because the automatic scaling
algorithm failed, but this seems unrelated to the problem)
WIPER is not a good substitute because in a time plot it does not
expand data within our many short scans due to long gaps in between
them.
I realize that the most likely answer is an error on my part, but
cannot find it.
Any Ideas?
With thanks, Barney (Rickett)
On Feb 17, 2009, at 2:46 PM, Amy Mioduszewski wrote:
> I think it is likely that you have too many rows in your FG table.
> How many do you have from the "copied*****" I assume it is more
> than 100000. That seems like a lot anyway. The easiest to test
> this (and a work around if too many flags is the problem) is to do
> a UVCOP and apply the flags then run EDITR on the output of UVCOP
> and see if you can add more flags.
>
> Amy
>
> Barney Rickett wrote:
>> Dear AIPS helper (I am hoping the daip address is still correct),
>> Please can you advise me on the following problem.
>> I am flagging a large data set from project AL730 recorded in
>> January this year and have difficulties with 'EDITR'.
>> The data consists of 3sec samples continuum data with a 50 sec
>> scan on a set of about 160 sources using two subarrays with IFs
>> AC at 6.6 GHz and BD at 4.8 GHz. The observations continued for
>> about 109 hrs (!) and have been concatenated into a single large
>> file.
>> I am working on data from one of the subarrays and analyzing IF1.
>> Initial tasks are TACOP to make a copy of the CL table 1, QUACK to
>> remove first 10sec of each scan, SETJY to set the flux density scale.
>> I then used EDITA to flag a substantial number of points based on
>> TANT values from the TY table that were above ~300 K (some were
>> in Giga Kelvin from a few of the antennas) creating FG2.
>> I then have been working through the source list flagging with
>> editr and looking at the results with uvplt. After the use of
>> edita, I set FLAGVER to 0 and was appending the flags to FG2 by
>> setting OUTFGVER 2 also. (I also tried with FLAGVER 2). After a
>> bout of flagging I EXIT and look at that source with uvplt set
>> from amplitude vs UVbaseline length. After flagging some sources
>> successfully (primary calibrator 3C286 and some secondary
>> calibrators) I now find that newly stored flags do not seem to be
>> entered into the FG table (or not recognized by UVPLT). I tried
>> allowing the FG table number to increase by setting OUTFGVER 0
>> and FLAGVER 0, which I thought solved the problem initially but
>> does not work now.
>> Also I rerun editr with flagver set to read the previous flags
>> but see plots with the previously flagged points still
>> displayed. I reflag them and repeat the sequence and get the
>> same result, even though a REPLOT in EDITR removes the flagged
>> pts from the plot. I also tried with and without calibration -
>> setting DOCALIB -1 1 2, with no noticeable change.
>> I have tried restarting the whole sequence several times but
>> always end up with unflaggable points. As a relative novice to
>> AIPS I assumed I was making errors in the setup, but after
>> several weeks of attempts I am at a loss as to how to proceed.
>> I wonder if there is some limit to the number of entries into a
>> flag table?
>> Can you suggest any diagnostic or change in procedure?
>> I am appending the inputs and outputs from EDITR that might
>> provide clues as to what I am doing wrong.
>> Here is the typical inp to 'editr' from a recent run
>> >inp
>> AIPS 1: EDITR: Interactive baseline-oriented UV data editor using
>> the TV
>> AIPS 1: Adverbs Values Comments
>> AIPS 1:
>> ----------------------------------------------------------------
>> AIPS 1: INNAME 'AL730_CONC' Input UV data (name)
>> AIPS 1: INCLASS 'DBCON' Input UV data (class)
>> AIPS 1: INSEQ 1 Input UV data (seq. #)
>> AIPS 1: INDISK 1 Input UV data disk drive #
>> AIPS 1: SOURCES 'G01183910' Source list
>> AIPS 1: *rest ' '
>> AIPS 1: QUAL -1 Calibrator qualifier
>> -1=>all
>> AIPS 1: CALCODE ' ' Calibrator code '
>> '=>all
>> AIPS 1: TIMERANG *all 0 Time range to use
>> AIPS 1: SELBAND -1 Bandwidth to select (kHz)
>> AIPS 1: SELFREQ -1 Frequency to select (MHz)
>> AIPS 1: FREQID 1 Freq. ID to select.
>> AIPS 1: SUBARRAY 0 Subarray, 0=>1
>> AIPS 1: DOCALIB -1 > 0 calibrate data &
>> weights
>> AIPS 1: > 99 do NOT calibrate
>> weights
>> AIPS 1: GAINUSE 2 CL (or SN) table to apply
>> AIPS 1: DOPOL -1 If >0 correct
>> polarization.
>> AIPS 1: BLVER -1 BL table to apply.
>> AIPS 1: FLAGVER 5 Flag table version
>> AIPS 1: OUTFGVER 0 Output FG table version
>> AIPS 1: DOBAND -1 If >0 apply bandpass cal.
>> AIPS 1: Method used depends on
>> value
>> AIPS 1: of DOBAND (see HELP file).
>> AIPS 1: BPVER -1 Bandpass table version
>> AIPS 1: SMOOTH *all 0 Smoothing function. See
>> AIPS 1: HELP SMOOTH for details.
>> AIPS 1: STOKES ' ' Use 'I' to edit RR and LL
>> AIPS 1: combined, else task
>> will set
>> AIPS 1: BCHAN 1 Low freq. channel in
>> average
>> AIPS 1: ECHAN 1 High freq channel in
>> average
>> AIPS 1: BIF 1 First IF displayed
>> AIPS 1: EIF 1 Last IF displayed
>> AIPS 1: UVRANGE 0 0 Min & max baseline
>> (klambda)
>> AIPS 1: ANTENNAS 2 3 List of antennas to be
>> used
>> AIPS 1: 4 9 11 13
>> AIPS 1: 16 17 18 21
>> AIPS 1: 24 26 28 *rest 0
>> AIPS 1: IN2NAME ' ' 2nd UV file name
>> AIPS 1: IN2CLASS ' ' 2nd UV file class
>> AIPS 1: IN2SEQ 0 2nd UV file sequence
>> AIPS 1: IN2DISK 0 2nd UV file disk
>> AIPS 1: DOHIST -1 Record flags in history
>> file
>> AIPS 1: SOLINT 0 Data averaging time in
>> mins
>> AIPS 1: DETIME 0 Break interval (min)
>> AIPS 1: DOWEIGHT 1 Rescale weights by
>> DOWEIGHT
>> AIPS 1: DOTWO 1 True => do second
>> observable
>> AIPS 1: plot of main baseline
>> AIPS 1: EXPERT 0 > 0 start in expert mode
>> AIPS 1: CROWDED 0 > 0 => allow plots with
>> all
>> (Most editr runs had flagver 0 outfgver 2)
>> Here is a typical output from MSGSERV:
>> I dont understand the warning about an FC table
>> I dont understand the message "Doing no flagging this time"
>> LOCALH> EDITR1: Task EDITR (release of 31DEC09) begins
>> LOCALH> EDITR1: Doing no flagging this time
>> LOCALH> EDITR1: **** WARNING: USING A PRE-EXISTING FC TABLE ****
>> LOCALH> EDITR1: DOING SUBARRAY 1 ONLY
>> LOCALH> EDITR1: Read the data to list times
>> LOCALH> EDITR1: Allocate 311609 words at 5562814
>> LOCALH> EDITR1: All data will reside in memory
>> LOCALH> EDITR1: Found 361 times, stored on a time axis of length
>> 415
>> LOCALH> EDITR1: Read the data into memory
>> LOCALH> EDITR1: Plotting:
>> LOCALH> EDITR1: Press buttons A, B, or C to choose an operation
>> LOCALH> EDITR1: Press button D for on-line help
>> ....
>> .... etc
>> .....
>> LOCALH> EDITR1: Flagged 151 points in T x F 1 414 x 1.360E
>> +00 1.000E+20
>> LOCALH> EDITR1: Press buttons A, B, or C to choose an operation
>> LOCALH> EDITR1: Press button D for on-line help
>> LOCALH> EDITR1: Plotting:
>> LOCALH> EDITR1: Press buttons A, B, or C to choose an operation
>> LOCALH> EDITR1: Press button D for on-line help
>> LOCALH> EDITR1: EDFCAP: copied***** rows from FG table version 4
>> LOCALH> EDITR1: EDFCAP: added 152 rows to FG table version 5
>> LOCALH> EDITR1: Appears to have ended successfully
>> LOCALH> EDITR1: localhos 31DEC09 TST: Cpu= 46.3 Real=
>> 170 IO= 59
>> _______________________________________________
>> Daip mailing list
>> Daip at listmgr.cv.nrao.edu
>> http://listmgr.cv.nrao.edu/mailman/listinfo/daip
More information about the Daip
mailing list