CFITSIO 2.0 beta

[William Pence]

CFITSIO 2.0 (Beta) Release

This is to announce the first public beta release of CFITSIO 2.0, the
interface library for reading and writing FITS files from C or Fortran
programs.  This new release is backward compatible with the previous
v1.42 release, but contains many new features, including:

 - FITS files can be directly accessed over the internet by
   providing the full FTP or HTTP URL to CFITSIO's open_file routine.

 - FITS files may be accessed in Shared Memory instead of on magnetic
   disk, to improve data processing speeds.

 - Input FITS files can be filtered or modified at run-time based on
   a user-supplied expression that is appended to the name of
   the input file.  This filtering operation is transparent to the
   application program that opens the file.

 - New routines are provided to help deal with large sets of
   hierarchically grouped FITS files.

 - Fortran-callable wrappers are provided, so the CFITSIO routines
   may be called just as easily from Fortran as well as from C.


The ability to filter an input FITS file on the fly at run-time is
perhaps the most significant new feature, and provides sophisticated
data preprocessing capabilities to every application that uses CFITSIO
to read FITS files.  For example, if one has a program that opens an
input file whose name is supplied by the user, then the file as seen by
the program may be modified by appending various qualifiers to the file
name.

Example 1: 'myfile.fits.gz[EVENTS][TIME > 4000]'

  CFITSIO will open and uncompress the gzipped file, move to the table
  extension which has EXTNAME = 'EVENTS', and then filter out any rows
  which have a TIME column value less than or equal to 4000.  So instead
  of opening the original file, the application program actually opens a
  copy of the file that contains only the selected rows in the EVENTS
  extension. (The original file is not modified).

Example 2: 'myfile.fits.gz[EVENTS][col PI = PHA * 1.1 + .3][TIME > 4000]'

  Similar to the first example, except it also creates on the fly a new
  table column called 'PI' which is a function of the existing column
  'PHA'.

Example 3: 'myfile.fits.gz[EVENTS][bin (x,y)]

  In this case, CFITSIO creates a 2-D image by binning (histogramming)
  the values in the X and Y columns in the EVENTS table extension.  The
  application program then opens this temporary 2-D image FITS file that
  has been created on the fly, rather than the original FITS table.

These are only very simple examples of the allowed filtering syntax,
which essentially supports arbitrarily complex expressions using a C or
Fortran-like syntax.  These filtering operations are also extremely
fast, and in many cases the CPU overhead to do the filtering is
insignificant compared to the time required to simply read the file on
disk (with typically I/O rates of 5 MB/s or greater).

This beta version of CFITSIO is available via a link on the FITSIO Home
Page at:

     http://heasarc.gsfc.nasa.gov/fitsio

A html version of the new CFITSIO 2.0 User's Guide may be viewed on
line from that site.  The Chapter entitled "Detailed Filename Syntax"
in particular describes the available file filtering and preprocessing
commands in CFITSIO.

Since this is a beta release, it is expected that new updates will be
released at approximately 2-week intervals, until the full release in
September 1998.

Acknowledgements:

Many people have contributed to the new capabilities in CFITSIO 2.0.
Jurek Borkowski, Bruce O'Neel, and Don Jennings from the Integral
Science Data Center, Switzerland, provided the initial design for the
plug-in I/O drivers that are now used by CFITSIO.  This `driver'
concept greatly simplified  the low-level I/O, which in turn made many
of the other new features in CFITSIO much easier to implement.  Jurek
Borkowski then wrote the Shared Memory driver, and Bruce O'Neel wrote
the drivers for accessing FITS files over the network using the FTP,
HTTP, and ROOT protocols.  Don Jennings also designed and wrote the
Hierarchical Grouping routines.

Uwe Lamers (XMM project at ESTEC, The Netherlands) designed the
high-performance lexical parsing algorithm that is used to do
on-the-fly filtering of FITS tables.  This algorithm essentially
pre-compiles the user-supplied expression into a form that can be
efficiently evaluated for each row.  Peter Wilson (RSTX, NASA/GSFC)
then combined this design with the CFITSIO iterator routine to further
enhance the data processing throughput.

Peter Wilson also wrote the set of Fortran-callable wrappers for all
the CFITSIO routines, which rely on the CFORTRAN macro developed by
Burkhard Burow.

____________________________________________________________________
Dr. William Pence                          pence at tetra.gsfc.nasa.gov
NASA/GSFC Code 662         HEASARC         +1-301-286-4599 (voice)     
Greenbelt MD 20771                         +1-301-286-1684 (fax)