[Difx-users] VDIF data with sample granularity = 2

[江悟]

Hi Adam,


Thanks a lot! It is clear now.


Cheers,
Wu


-----原始邮件-----
发件人:"Adam Deller" <adeller at gmail.com>
发送时间:2018-12-13 18:45:26 (星期四)
收件人: "江悟" <jiangwu at shao.ac.cn>
抄送: difxusers <difx-users at listmgr.nrao.edu>
主题: Re: [Difx-users] VDIF data with sample granularity = 2


Hi Wu Jiang,


No, data will not be skipped.  sample granularity is an internal concept for the mark 5 data stream, which tells it that it needs to treat the samples in blocks of "samplegranularity" wide together in order to unpack correctly.  I misspoke earlier when I said the issue was integer number of nanoseconds, that is a different granularity (framegranularity, if you care :) ).  samplegranularity effectively refers to how many different time samples are packed into the single unit of data that is unpacked at a time (typically 1 byte).  If you have e.g. a datastream with 4x2 bit samples, then all the samples from a single time are packed together into 8 bits, and doing integer arithmetic on a byte pointer will always get you to the right place to start unpacking.  But if you only have 2x2 bit samples in the stream, then that's only 4 bits altogether per time, and there are two different times that would point to the same byte (one wants the first nibble, one wants the second nibble).  Hence the concept of sample granularity.


Cheers,
Adam


On Thu, 13 Dec 2018 at 19:47, 江悟 <jiangwu at shao.ac.cn> wrote:



Hi Adam and all,


As I discussed with Adam already, I still not sure if some part of the raw data would be skipped to load for correlation when sample granularity = 2. Any comments?


Cheers,
Wu


-----原始邮件-----
发件人:"Adam Deller" <adeller at gmail.com>
发送时间:2018-12-13 15:22:30 (星期四)
收件人: "江悟" <jiangwu at shao.ac.cn>
抄送:
主题: Re: VDIF data with sample granularity = 2


HI Wu Jiang,


Can you please send emails like this to the difx-users mailing list, rather than to me direct?  sample granularity 2 means that the data reader will only see a timestamp with an integer number of nanoseconds every second time it unpacks a group of samples.  This is because of the very high sampling rate.


Cheers,
Adam






On Thu, 13 Dec 2018 at 11:43, 江悟 <jiangwu at shao.ac.cn> wrote:

Dear Adam,

Recently when I made correlations with VDIF data streams, DiFX decoded the data format and gave some information as follows.
Mark5 stream: 0x2e2dc80
  stream = File-1/1=/Bgfsdata/astro/raw/fa02/scan1.source.1045.vdif
  format = VDIF_4096-6400-2-2 = 3
  start mjd/sec = 58454 40673.694666240
  frame duration =  5120.00 ns
  framenum = 0
  sample rate = 1600000000 Hz
  offset = 0
  framebytes = 4128 bytes
  datasize = 4096 bytes
  sample granularity = 2
  frame granularity = 1
  gframens = 5120
  payload offset = 32
  read position = 0
  data window size = 1048576 bytes
Where,  I am not quite sure about "the sample granularity = 2", as always I just see "sample granularity = 1" in many cases. Does it mean DiFX recognizes the samples in the data stream are not continuously allocated? Such as sample_ch1 sample_ch2 x x sample_ch1 sample_ch2 x x. However, actually the samples in the data stream are continuously recorded. So I would like to make clear on what DiFX deals with the data?
Thank you!

Best regards,
Wu Jiang





--

!=============================================================!
Dr. Adam Deller         
ARC Future Fellow, Senior Lecturer
Centre for Astrophysics & Supercomputing 
Swinburne University of Technology    
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
_______________________________________________
Difx-users mailing list
Difx-users at listmgr.nrao.edu
https://listmgr.nrao.edu/mailman/listinfo/difx-users





--

!=============================================================!
Dr. Adam Deller         
ARC Future Fellow, Senior Lecturer
Centre for Astrophysics & Supercomputing 
Swinburne University of Technology    
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20181213/8363ff98/attachment-0001.html>