[Difx-users] VDIF data with sample granularity = 2

[Adam Deller]

Hi Wu Jiang,

No, data will not be skipped.  sample granularity is an internal concept
for the mark 5 data stream, which tells it that it needs to treat the
samples in blocks of "samplegranularity" wide together in order to unpack
correctly.  I misspoke earlier when I said the issue was integer number of
nanoseconds, that is a different granularity (framegranularity, if you care
:) ).  samplegranularity effectively refers to how many different time
samples are packed into the single unit of data that is unpacked at a time
(typically 1 byte).  If you have e.g. a datastream with 4x2 bit samples,
then all the samples from a single time are packed together into 8 bits,
and doing integer arithmetic on a byte pointer will always get you to the
right place to start unpacking.  But if you only have 2x2 bit samples in
the stream, then that's only 4 bits altogether per time, and there are two
different times that would point to the same byte (one wants the first
nibble, one wants the second nibble).  Hence the concept of sample
granularity.

Cheers,
Adam

On Thu, 13 Dec 2018 at 19:47, 江悟 <jiangwu at shao.ac.cn> wrote:

>
> Hi Adam and all,
>
> As I discussed with Adam already, I still not sure if some part of the raw
> data would be skipped to load for correlation when sample granularity = 2.
> Any comments?
>
> Cheers,
> Wu
>
> -----原始邮件-----
> *发件人:*"Adam Deller" <adeller at gmail.com>
> *发送时间:*2018-12-13 15:22:30 (星期四)
> *收件人:* "江悟" <jiangwu at shao.ac.cn>
> *抄送:*
> *主题:* Re: VDIF data with sample granularity = 2
>
> HI Wu Jiang,
>
> Can you please send emails like this to the difx-users mailing list,
> rather than to me direct?  sample granularity 2 means that the data reader
> will only see a timestamp with an integer number of nanoseconds every
> second time it unpacks a group of samples.  This is because of the very
> high sampling rate.
>
> Cheers,
> Adam
>
>
>
> On Thu, 13 Dec 2018 at 11:43, 江悟 <jiangwu at shao.ac.cn> wrote:
>
>> Dear Adam,
>>
>> Recently when I made correlations with VDIF data streams, DiFX decoded
>> the data format and gave some information as follows.
>> Mark5 stream: 0x2e2dc80
>>   stream = File-1/1=/Bgfsdata/astro/raw/fa02/scan1.source.1045.vdif
>>   format = VDIF_4096-6400-2-2 = 3
>>   start mjd/sec = 58454 40673.694666240
>>   frame duration =  5120.00 ns
>>   framenum = 0
>>   sample rate = 1600000000 Hz
>>   offset = 0
>>   framebytes = 4128 bytes
>>   datasize = 4096 bytes
>>   sample granularity = 2
>>   frame granularity = 1
>>   gframens = 5120
>>   payload offset = 32
>>   read position = 0
>>   data window size = 1048576 bytes
>> Where,  I am not quite sure about "the sample granularity = 2", as always
>> I just see "sample granularity = 1" in many cases. Does it mean DiFX
>> recognizes the samples in the data stream are not continuously allocated?
>> Such as sample_ch1 sample_ch2 x x sample_ch1 sample_ch2 x x. However,
>> actually the samples in the data stream are continuously recorded. So I
>> would like to make clear on what DiFX deals with the data?
>> Thank you!
>>
>> Best regards,
>> Wu Jiang
>>
>
>
> --
> !=============================================================!
> Dr. Adam Deller
> ARC Future Fellow, Senior Lecturer
> Centre for Astrophysics & Supercomputing
> Swinburne University of Technology
> John St, Hawthorn VIC 3122 Australia
> phone: +61 3 9214 5307
> fax: +61 3 9214 8797
>
> office days (usually): Mon-Thu
> !=============================================================!
>
> _______________________________________________
> Difx-users mailing list
> Difx-users at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>


-- 
!=============================================================!
Dr. Adam Deller
ARC Future Fellow, Senior Lecturer
Centre for Astrophysics & Supercomputing
Swinburne University of Technology
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20181213/092df26f/attachment.html>