[Difx-users] structure of fits-idi files

[harro verkouter]

Hi Adam,

That is exactly how things should work. But probably for the FITS-IDI files
from DiFX it's not entirely true. I know that we (=JIVE/SFXC) write all the
MAXIS, MAXISn and CTYPEn keywords and this allows astropy.fits to correctly
decompose the matrix column elements into properly shaped n-dimensional
arrays instead of one big blob per row as from a DiFX FITS file.

To me it is clear that some keywords are missing from the DiFX FITS-IDI
that prevent the FITS library from doing that; that's what Franz is seeing.

Cheers,
h

On 26 October 2016 at 09:17, Adam Deller <adeller at astro.swin.edu.au> wrote:

> Hi Franz,
>
> I looked up the FITS-IDI standard, and I think it describes how to extract
> this information.  See Section 4.1 of http://www.aoc.nrao.edu/~egrei
> sen/AIPSMEM114.PDF.  Basically, you need to look at the CTYPE# keywords,
> where # runs from 1 through to 5 or 6.  CTYPE1 describes the fastest
> varying parameter, CTYPE2 the next fastest, and so on.  Corresponding
> keywords MAXIS# give the number of entries along that axis.  CTYPE1 is
> constrained to be of type COMPLEX (i.e., the re/im[/weight] values) but the
> other ones can come in whatever order, you need to look at the keywords.
> They are STOKES for polarisation, FREQ for frequency channel, and BAND for
> the subband.
>
> These should all be accessible via astropy.
>
> Cheers,
> Adam
>
> On 25 October 2016 at 17:57, harro verkouter <hverkouter at gmail.com> wrote:
>
>>
>> Hi Franz,
>>
>> FITS dates from the era of FORTRAN and thus the first axis should go
>> fastest, then the 2nd etc.
>> But it depends a bit on what the (implied) order of the axes is in the
>> difx FITS-IDI array. You may want to take a closer look at the full header
>> for the 'UV_DATA' table - there may be hints in there what the order of the
>> axes is within the 'FLUX' column. The logical choice would indeed be IF *
>> CHANNELS * POL * 2 values / complex number (or 3, in sfxc's case; we record
>> a weight per spectral point too: (re, im, wgt)), thus IF goes first, then
>> channel etc.
>>
>> In sfxc those axes have been made explicit - as you can tell, so there's
>> no ambiguity of how to interpret them.
>>
>> Cheers,
>> h
>>
>>
>> On 25 October 2016 at 06:28, Franz Kirsten <franz.kirsten at curtin.edu.au>
>> wrote:
>>
>>> Hi all,
>>>
>>> I am currently looking at the fits-idi files as they are spit out by
>>> DiFX -- and noticed a difference in the structure of the visibility array
>>> compared to that produced by SFXC**. I'd mostly like to know how those
>>> visibilities are structured/ordered in the 'FLUX' field of the UV_DATA
>>> table.
>>>
>>> Just so you know what I look at (in python):
>>> import astropy.io.fits as fits
>>> hdulist=fits.open('./difx-file.idifits')
>>> vis=hdulist['UV_DATA'].data['FLUX']
>>> vis.shape
>>> > (n_times * n_baselines, some_number)
>>>
>>> Now, in my case, some_number = 4096. I have 8 IFs, 128 channels/IF and 2
>>> polarisations. Assuming I have a pair of numbers (amp + phase, or
>>> real+imaginary?) for each visibility, 8 * 128 * 2 * 2 = 4096. But how are
>>> the data ordered? Are the, say, first 256 entries amp+phase for IF1,
>>> chan1-chan128, pol 1? In this logic the next 256 entries would be that of
>>> IF1, chan1-128, pol2. And then I'd jump to IF2 and repeat. Is this correct?
>>>
>>> Many thanks for your help!
>>>
>>> Regards,
>>> Franz
>>>
>>> **In fits-idi files that come out of SFXC:
>>> vis.shape = (n_times * n_baselines, 1, 1, n_IF, n_chan, n_pol, 3), where
>>> the three entries in the last index are amp, phase, weight.
>>>
>>> _______________________________________________
>>> Difx-users mailing list
>>> Difx-users at listmgr.nrao.edu
>>> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>>>
>>
>>
>> _______________________________________________
>> Difx-users mailing list
>> Difx-users at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/difx-users
>>
>>
>
>
> --
> !=============================================================!
> Dr. Adam Deller
> ARC Future Fellow, Senior Lecturer
> Centre for Astrophysics & Supercomputing
> Swinburne University of Technology
> John St, Hawthorn VIC 3122 Australia
> phone: +61 3 9214 5307
> fax: +61 3 9214 8797
> !=============================================================!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20161026/9f88a2a5/attachment-0001.html>