[daip] questions about rflag and flag tables

[Eric Greisen]

Susan Neff wrote:
> On 7/24/12 4:13 PM, Eric Greisen wrote:
>> Susan Neff wrote:
>>> Hi Eric,
>>>
>>> I have a question about RFLAG and flag tables.
>>>
>>> What I am doing is as follows:
>>> The first time I run RFLAG, I give it
>>> flagver = xx, outfgver = 0,
>>> so it will create a new flag table by
>>> copying flagver xx and then adding to it.
>>> I run rflag on a subset of sources,
>>> creating new flag table yy.
>>> Then I try running rflag again on
>>> different sources, using flagver = xx
>>> and outfgver = yy.
>>>
>>> What happens to the flags created in the
>>> run on the second source??
>>>
>>> I'm asking because it seems that the flags
>>> produced for the second source are not making
>>> it into the flag table.  The output plots
>>> I get by running with doplot = -12 look
>>> like it got the flags, and the noise
>>> and scut returned adverbs look like
>>> they got the flags.  However, if I try to
>>> run another round of rflag, on the second
>>> source, using flagtable yy,  I get
>>> the same starting noise and scut values
>>> as I had in the first run on the second
>>> source... not lower values as I would expect.
>>> (A second run on the first source behaves
>>> as I expect - i.e. the second run returns
>>> much lower values for noise(i) and scut(i).
>>>
>>> I'm wondering if the flags for the
>>> second sources are not
>>> actually making it into the output
>>> flag table ( if it is specified, as opposed
>>> to being set to zero and therefore
>>> creating a new table).  I had
>>> thought they would be added into
>>> the flag table created with the first source, but
>>> now I'm wondering if I have misinterpreted
>>> what is happening.
>>>
>>> Do I always need to create a new flag
>>> table to capture the the newly created
>>> flags?  And if I create a new flag table,
>>> using only some of the sources, does
>>> it carry along the flags for all sources?
>> I am back and will look at this for you.  Waht I do know is that when
>> you run with DOPLOT=-12, the program uses the flag table that it just
>> wrote.  So if the plots at that point look good, then that flag table
>> must contain the new flags.  The task cannot use FLAGVER=xxx,
>> OUTFGVER=xxx since we cannot add to an FG table that we are reading.  In
>> that case it writes a new one but should then fix it at the end.  I will
>> try things and see what happens.
>>
>> Eric
> Hi Eric,
> 
> Welcome back and thank you.  I've been moving ahead by
> just writing a new flag table each time, as Amy suggested.
> That works fine.   After a few runs, I run UVCOP and apply
> the flags, so I don't build up huge flag tables.  
> (Given my
> highly interrupted schedule here, it isn't that big a hit
> on my time, as long as I can start a UVCOP before going
> off to whatever meeting has suddenly materialized).
> 
> So my question is, if I run with flagver = xxx, and 
> outfgver = 0
> (which creates flagversion xxx+ 1), can I then change my
> inputs (source, noise values, whatever), run rflag again
> using flagver= xxx and outfgver=xxx+1 and get both
> sets of flags combined in the flagtable xxx+1 ?

The handling of flag versions was, I think, confused in the task and I 
have clarified it:

13897.  July 25, 2012             RFLAG                 Eric
         Adjusted the handling of FG table versions.  New flags are
         written to OUTFGVER directly unless OUTFGVER=FLAGVER.  Old
         flags from FLAGVER are copied to OUTFGVER if and only if
         OUTFGVER represents a new table.  If FLAGVER=OUTFGVER (after
         application of defaults), the task uses a temporary FG table
         filled with the old flags and any new ones and then replaces
         the old FG ver with the temporary one.
         Moved nowhere.

When you ran on the 2nd source reading xxx and writing yyy, the plots 
done after the flagging with DOPLOT=-12 used yyy as FLAGVER and gave you 
desirable results.  IF you changed FLAGVER to yyy, those results should 
have remained in place and NOISE and SCUT should have gone down some 
more.  One of the confusions in the old version involved 
FLAGVER=OUTFGVER=yyy which the program cannot actually handle without 
some magic.  It tried to do that but I can see ways in which it would 
get confused.  The new one does this fine.

Setting FLAGVER=OUTFGVER=0 is still the easiest thing to do.

A MNJ should be run tomorrow morning.

Eric Greisen