[daip] Fwd: Fwd: Use of EDITR in AL730 project
Barney Rickett
bjrickett at ucsd.edu
Mon Feb 23 20:26:09 EST 2009
Sorry forgot the attachment:
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: editr_2.txt
URL: <http://listmgr.nrao.edu/pipermail/daip/attachments/20090223/15661efc/attachment.txt>
-------------- next part --------------
Barney
Begin forwarded message:
> From: Barney Rickett <bjrickett at ucsd.edu>
> Date: February 23, 2009 5:24:09 PM PST
> To: daip at nrao.edu
> Cc: Amy Mioduszewski <amiodusz at nrao.edu>, Lucyna Kedziora-Chudczer
> <Lucyna.Kedziora-Chudczer at csiro.au>, Jim Lovell
> <Jim.Lovell at utas.edu.au>, Roopesh Ojha <Roopesh.Ojha at gmail.com>,
> Vivek Dhawan <vdhawan at aoc.nrao.edu>
> Subject: Re: [daip] Fwd: Use of EDITR in AL730 project
>
> Dear DAIP & Amy (not sure to whom I should send this plea for help)
>
> I have followed the suggestion by Amy - but still cannot get editr
> to add flags for my uv data... as I described in my original
> email below.
>
> I used extd and tacop to copy my highest FG table to FG1 and then
> used uvcop to create a new catalogued file.
>
> I then used edita to flag the many 'ty' values that were above 300K
> in the copied file. On rerunning edita the high ty are not shown -
> evidently edita worked OK. I then tried editr, uvplt and wiper on
> data from one secondary calibrator source (of our multi-source
> data) in various ways.
> I found that, even though editr seems to exit successfully and
> msgsrv states that rows were added to my FG table, the points I had
> flagged keep coming back in both editr and uvplt.
> However, when I used 'wiper' I was able to flag points and they did
> not reappear in uvplt or editr. I conclude that wiper is working
> for me but editr is not.
>
> I then restarted editr on the data with the FG version generated by
> wiper. I flagged another clear set of bad points and saw that FG
> had gone from 5 to 6, but these points are still present in uvplt
> with flagver = 6 (and also in a re-run of editr).
>
> I am attaching a text file with the inp to editr, the state of
> headers from imh and the output on msgsrv during the use of editr.
>
> I aplogise for bothering you all with my difficulties but I am
> stymied at present. After very many hours trying to edit the 5
> days of data from the VLA, I cannot make the flagging work with
> EDITR. (I see that the number of rows in the FG table is now only
> 179. I removed all data from one antenna and see several messages
> "EDTINC: TIC MARK ALGORITHM FAILED! CONTINUING." because the
> automatic scaling algorithm failed, but this seems unrelated to the
> problem)
>
> WIPER is not a good substitute because in a time plot it does not
> expand data within our many short scans due to long gaps in between
> them.
>
> I realize that the most likely answer is an error on my part, but
> cannot find it.
>
> Any Ideas?
> With thanks, Barney (Rickett)
>
>
>
> On Feb 17, 2009, at 2:46 PM, Amy Mioduszewski wrote:
>
>> I think it is likely that you have too many rows in your FG
>> table. How many do you have from the "copied*****" I assume it is
>> more than 100000. That seems like a lot anyway. The easiest to
>> test this (and a work around if too many flags is the problem) is
>> to do a UVCOP and apply the flags then run EDITR on the output of
>> UVCOP and see if you can add more flags.
>>
>> Amy
>>
>> Barney Rickett wrote:
>>> Dear AIPS helper (I am hoping the daip address is still correct),
>>> Please can you advise me on the following problem.
>>> I am flagging a large data set from project AL730 recorded in
>>> January this year and have difficulties with 'EDITR'.
>>> The data consists of 3sec samples continuum data with a 50 sec
>>> scan on a set of about 160 sources using two subarrays with IFs
>>> AC at 6.6 GHz and BD at 4.8 GHz. The observations continued for
>>> about 109 hrs (!) and have been concatenated into a single large
>>> file.
>>> I am working on data from one of the subarrays and analyzing IF1.
>>> Initial tasks are TACOP to make a copy of the CL table 1, QUACK to
>>> remove first 10sec of each scan, SETJY to set the flux density
>>> scale.
>>> I then used EDITA to flag a substantial number of points based
>>> on TANT values from the TY table that were above ~300 K (some
>>> were in Giga Kelvin from a few of the antennas) creating FG2.
>>> I then have been working through the source list flagging with
>>> editr and looking at the results with uvplt. After the use of
>>> edita, I set FLAGVER to 0 and was appending the flags to FG2 by
>>> setting OUTFGVER 2 also. (I also tried with FLAGVER 2). After
>>> a bout of flagging I EXIT and look at that source with uvplt set
>>> from amplitude vs UVbaseline length. After flagging some
>>> sources successfully (primary calibrator 3C286 and some
>>> secondary calibrators) I now find that newly stored flags do not
>>> seem to be entered into the FG table (or not recognized by
>>> UVPLT). I tried allowing the FG table number to increase by
>>> setting OUTFGVER 0 and FLAGVER 0, which I thought solved the
>>> problem initially but does not work now.
>>> Also I rerun editr with flagver set to read the previous flags
>>> but see plots with the previously flagged points still
>>> displayed. I reflag them and repeat the sequence and get the
>>> same result, even though a REPLOT in EDITR removes the flagged
>>> pts from the plot. I also tried with and without calibration -
>>> setting DOCALIB -1 1 2, with no noticeable change.
>>> I have tried restarting the whole sequence several times but
>>> always end up with unflaggable points. As a relative novice to
>>> AIPS I assumed I was making errors in the setup, but after
>>> several weeks of attempts I am at a loss as to how to proceed.
>>> I wonder if there is some limit to the number of entries into a
>>> flag table?
>>> Can you suggest any diagnostic or change in procedure?
>>> I am appending the inputs and outputs from EDITR that might
>>> provide clues as to what I am doing wrong.
>>> Here is the typical inp to 'editr' from a recent run
>>> >inp
>>> AIPS 1: EDITR: Interactive baseline-oriented UV data editor using
>>> the TV
>>> AIPS 1: Adverbs Values Comments
>>> AIPS 1:
>>> ----------------------------------------------------------------
>>> AIPS 1: INNAME 'AL730_CONC' Input UV data (name)
>>> AIPS 1: INCLASS 'DBCON' Input UV data (class)
>>> AIPS 1: INSEQ 1 Input UV data (seq. #)
>>> AIPS 1: INDISK 1 Input UV data disk
>>> drive #
>>> AIPS 1: SOURCES 'G01183910' Source list
>>> AIPS 1: *rest ' '
>>> AIPS 1: QUAL -1 Calibrator qualifier
>>> -1=>all
>>> AIPS 1: CALCODE ' ' Calibrator code '
>>> '=>all
>>> AIPS 1: TIMERANG *all 0 Time range to use
>>> AIPS 1: SELBAND -1 Bandwidth to select (kHz)
>>> AIPS 1: SELFREQ -1 Frequency to select (MHz)
>>> AIPS 1: FREQID 1 Freq. ID to select.
>>> AIPS 1: SUBARRAY 0 Subarray, 0=>1
>>> AIPS 1: DOCALIB -1 > 0 calibrate data &
>>> weights
>>> AIPS 1: > 99 do NOT calibrate
>>> weights
>>> AIPS 1: GAINUSE 2 CL (or SN) table to apply
>>> AIPS 1: DOPOL -1 If >0 correct
>>> polarization.
>>> AIPS 1: BLVER -1 BL table to apply.
>>> AIPS 1: FLAGVER 5 Flag table version
>>> AIPS 1: OUTFGVER 0 Output FG table version
>>> AIPS 1: DOBAND -1 If >0 apply bandpass cal.
>>> AIPS 1: Method used depends on
>>> value
>>> AIPS 1: of DOBAND (see HELP
>>> file).
>>> AIPS 1: BPVER -1 Bandpass table version
>>> AIPS 1: SMOOTH *all 0 Smoothing function. See
>>> AIPS 1: HELP SMOOTH for details.
>>> AIPS 1: STOKES ' ' Use 'I' to edit RR and LL
>>> AIPS 1: combined, else task
>>> will set
>>> AIPS 1: BCHAN 1 Low freq. channel in
>>> average
>>> AIPS 1: ECHAN 1 High freq channel in
>>> average
>>> AIPS 1: BIF 1 First IF displayed
>>> AIPS 1: EIF 1 Last IF displayed
>>> AIPS 1: UVRANGE 0 0 Min & max baseline
>>> (klambda)
>>> AIPS 1: ANTENNAS 2 3 List of antennas to be
>>> used
>>> AIPS 1: 4 9 11 13
>>> AIPS 1: 16 17 18 21
>>> AIPS 1: 24 26 28 *rest 0
>>> AIPS 1: IN2NAME ' ' 2nd UV file name
>>> AIPS 1: IN2CLASS ' ' 2nd UV file class
>>> AIPS 1: IN2SEQ 0 2nd UV file sequence
>>> AIPS 1: IN2DISK 0 2nd UV file disk
>>> AIPS 1: DOHIST -1 Record flags in
>>> history file
>>> AIPS 1: SOLINT 0 Data averaging time in
>>> mins
>>> AIPS 1: DETIME 0 Break interval (min)
>>> AIPS 1: DOWEIGHT 1 Rescale weights by
>>> DOWEIGHT
>>> AIPS 1: DOTWO 1 True => do second
>>> observable
>>> AIPS 1: plot of main baseline
>>> AIPS 1: EXPERT 0 > 0 start in expert mode
>>> AIPS 1: CROWDED 0 > 0 => allow plots
>>> with all
>>> (Most editr runs had flagver 0 outfgver 2)
>>> Here is a typical output from MSGSERV:
>>> I dont understand the warning about an FC table
>>> I dont understand the message "Doing no flagging this time"
>>> LOCALH> EDITR1: Task EDITR (release of 31DEC09) begins
>>> LOCALH> EDITR1: Doing no flagging this time
>>> LOCALH> EDITR1: **** WARNING: USING A PRE-EXISTING FC TABLE ****
>>> LOCALH> EDITR1: DOING SUBARRAY 1 ONLY
>>> LOCALH> EDITR1: Read the data to list times
>>> LOCALH> EDITR1: Allocate 311609 words at 5562814
>>> LOCALH> EDITR1: All data will reside in memory
>>> LOCALH> EDITR1: Found 361 times, stored on a time axis of
>>> length 415
>>> LOCALH> EDITR1: Read the data into memory
>>> LOCALH> EDITR1: Plotting:
>>> LOCALH> EDITR1: Press buttons A, B, or C to choose an operation
>>> LOCALH> EDITR1: Press button D for on-line help
>>> ....
>>> .... etc
>>> .....
>>> LOCALH> EDITR1: Flagged 151 points in T x F 1 414 x 1.360E
>>> +00 1.000E+20
>>> LOCALH> EDITR1: Press buttons A, B, or C to choose an operation
>>> LOCALH> EDITR1: Press button D for on-line help
>>> LOCALH> EDITR1: Plotting:
>>> LOCALH> EDITR1: Press buttons A, B, or C to choose an operation
>>> LOCALH> EDITR1: Press button D for on-line help
>>> LOCALH> EDITR1: EDFCAP: copied***** rows from FG table version 4
>>> LOCALH> EDITR1: EDFCAP: added 152 rows to FG table version 5
>>> LOCALH> EDITR1: Appears to have ended successfully
>>> LOCALH> EDITR1: localhos 31DEC09 TST: Cpu= 46.3 Real=
>>> 170 IO= 59
>>> _______________________________________________
>>> Daip mailing list
>>> Daip at listmgr.cv.nrao.edu
>>> http://listmgr.cv.nrao.edu/mailman/listinfo/daip
>
More information about the Daip
mailing list