[Difx-users] Treatment of invalid sample bits in DiFX

[Walter Brisken]

My understanding is that if there are any valid samples in an fft frame, 
the processing proceeds.  All in valid data is set to zero and the amount 
of good data processed is tracked and stored in the weight that comes with 
the data.  These weights are baseline-based.  After each sub-integration, 
the sub-int weights are multiplied for the two datastreams and accumulatd. 
This does not yield an exactly correct weight, but in cases where weights 
are close to 1 it does a very good job.

 	-Walter

On Mon, 1 Aug 2016, Lei Liu wrote:

> Dear All,
>
> I would like to ask a question about the treatment of invalid sample bits in DiFX:
>
> Assume data are sampled in one bit and are recorded in Mark5a and Mark5b format. Some data frames might get lost due to network transfer. Whatÿÿs more, the frame header will overwrite some sample bits (the first 160 bits for a 20000 bits frame for one track) in Mark5a format. Data in the lost frame or overwritten by frame header will be labeled as invalid.
>
> We know the integration period is composed of many FFT periods. For each FFT period, we can calculate the fraction of invalid sample bits among the whole FFT points. For example, if the number of invalid sample bits are 160, and FFT point number is set to 1024, the fraction is 160 / 1024 = 0.15625. My question is, it seems that in DiFX, there is a threshold of invalid sample fraction. If the fraction exceeds the threshold, this FFT period will be discarded and will not take part in integration. If my postulation is correct, I would like to know what is the fraction? Is there any default value?
>
> Thank you in advance!
>
> Best wishes,
> Lei