[Difx-users] GPU-DiFX project and requests

[Adam Deller]

Hi everyone,

Chris, Michael and I had a productive meeting this morning with the team
that will be working on the GPU-DiFX project, which will commence in around
5 weeks and run through to the end of the year.  During that time, I may
put out periodic requests to the community for actions in support of the
project, which will hopefully lead to benefits for everyone.  Here is the
first one!

The ADACS team are rightly concerned that they don't break any existing
functionality while re-factoring code and porting GPU kernels in.  To that
end, they are very keen to have a test suite with as much coverage as
possible, that they can run to show that nothing has broken.  This is a
massive opportunity for the DiFX community, because historically we've
always wanted to do better on regression testing prior to putting out
releases, and indeed upon general commits to trunk.  I expect that on the
completion of this project we will have something like the continuous
integration setup that we've long desired.

So: can I please ask the communities assistance with putting together a
more extensive test suite?  I have space to store the data (up to a few GB
per test set, maybe 10 GB max) on the Swinburne supercomputer ozstar.  The
ideal situation would be if you can provide me with a complete set
including vex, v2d, baseband data files, and an already-correlated .difx
output.  I'm happy to liaise to help flesh out v2d files where needed,
though, and we can of course produce the reference output from scratch if
necessary.

As can be seen on
https://www.atnf.csiro.au/vlbi/dokuwiki/doku.php/difx/datasets, the
currently available test sets cover the following data arrangements:

* real-sampled USB
* real-sampled LSB

I also have access to CODIF 2+2 bit complex DSB (lower).  At a minimum, it
would be nice to add:

* Complex sampled USB
* Complex sampled LSB
* Complex sampled DSB (positive)

For the GPU project the choice of format is less important, but still it
would be nice for general purposes to get coverage of VDIF, CODIF, Mark4,
Mark5B, and LBA.

These of course don't have to be all separate tests, if there are single
scans that have a mix of multiple record types on different antennas that
would be perfect to cut down on the issues.

As well as a "vanilla" setup corresponding to each test, it would then be
good to add additional configurations (v2ds) covering zoom bands, output
bands, multiple field centres, and pulsar processing.

This is obviously quite the list, so I don't expect to fill it out before
the project starts.  But if we could continue to add to it during the
remainder of 2020 that would be great.  I will soon have access to a google
drive folder where we'll be storing documentation related to the GPU-DIFX
project, so one of the first pages I'll create will be a test set listing.
I will share that around (along with any other relevant info) as soon as it
is available.

Cheers,
Adam

-- 
!=============================================================!
A/Prof. Adam Deller
ARC Future Fellow
Centre for Astrophysics & Supercomputing
Swinburne University of Technology
John St, Hawthorn VIC 3122 Australia
phone: +61 3 9214 5307
fax: +61 3 9214 8797

office days (usually): Mon-Thu
!=============================================================!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/difx-users/attachments/20200721/d7de83e4/attachment.html>