[fitsbits] start of Public Comment Period on compressed FITS image and tables

[Robert Siverd]

For reference, DS9 has supported the tiled image convention for years
(since ~2008 according to release notes). It appears to do so using CFITSIO
rather than a separate implementation. A DS9 developer would probably know
better.

Rob


On Wed, Jun 24, 2015 at 10:06 AM, van Nieuwenhoven, Richard <
Richard.vanNieuwenhoven at adesso.at> wrote:

> At nom-tam-fits we wan't to implement the compression's and have done so
> partly already. The
> biggest problem is that the standard does not provide test data, and in my
> opinion this is a  prerequisite for a standard like this.
> In fact a good standard delivers a reference implementation or a
> compliance test.
>
> I can see that a reference implementation is not possible in a community
> like this, but the delivery of
> test data would make the live of a developer 1000 fold easyer. Especially
> if it is available in levels, like separate data files for the different
> kinds of compression.
>
> Sure the compression makes fits more comple, but that's what access
> library's like ours are for. Especially when the fits files start to get as
> big as e few gigabytes...
>
>        Ritchie
> ________________________________________
> From: fitsbits [fitsbits-bounces at listmgr.nrao.edu] on behalf of Mark
> Taylor [m.b.taylor at bristol.ac.uk]
> Sent: Wednesday, June 24, 2015 5:45 PM
> To: FITS List
> Cc: Amit Kapadia
> Subject: Re: [fitsbits] start of Public Comment Period on compressed FITS
> image and tables
>
> On Wed, 24 Jun 2015, Lucio Chiappetti wrote:
>
> > On Wed, 24 Jun 2015, Mark Taylor wrote:
> >
> > > but three weeks is not enough time to attempt an implementation to
> check it
> > > does actually provide all the description required to implement these
> > > conventions.
> >
> > That statement reminds me interoperability testing is part of the
> criteria to
> > be considered by the EC when going to votes. I assumed the fact the
> registered
> > conventions (in general) have been around for quite a while could waive
> for
> > most of the interoperability demonstration.
>
> That does prove that it can be done and work well, but it doesn't prove
> that the way it's described in the text correctly and fully describes
> how it's done.
>
> > Nevertheless, while nobody is asking that topcat supports ZIMAGE and
> ZTABLE
> > now, in 3 weeks, 3 months or 3 years :-) it would be helpful if the
> original
> > authors and supporters of the convention may recall the status of
> > interoperability (multiple implementations), and also any other author of
> > popular s/w (e.g. ds9) may state whether they support it already.
>
> Yes, I'd be very interested to hear if there is more than one
> independent implementation out there.
>
> > > The text looks like what it presumably is, a post-hoc codification of a
> > > series of experiments in compression
> >
> > This seems a bit unfair or un-kind, but ...
>
> I apologise if that is or appears to be true.  The text itself looks
> well written and complete, but my point was that there is quite a
> large number of algorithmic options described, and my suspicion
> is that some of them may be there because they got implemented
> and then superceded by a better idea rather than because they
> are all useful alternatives.
>
> > > is all there in CFITSIO, but that can't be used directly by
> non-C-friendly
> > > languages such as java, javascript, and who knows
> >
> > is Fortran C-friendly ? or idl ?
>
> I would class IDL and FORTRAN as "C-friendly" on the grounds that
> they are (probably) implemented in C and can generally be linked
> cleanly to C routines.  Java is capable of interfacing with C
> via the Java Native Interface, but this breaks the 'pure java' model
> and greatly complicates distribution of java executables which
> are otherwise completely platform-independent, so I'd call it
> C-hostile.  In browser-based javascript I think it's generally
> impossible for security reasons to use C-based libraries or
> run C-based utilities.
>
> >                                  I would not engage myself in supporting
> > ZIMAGE or ZTABLE directly, not even in java ... I guess it won't be very
> > efficient.
>
> You might not, but I probably should.  If this becomes part of the
> standard (rather than an associated convention) I would hope to
> be able to support tile compressed tables in the java library STIL
> (hence STILTS/TOPCAT), though I'm not sure on what timescale.
> (There is no reason by the way that java cannot do this processing
> with comparable efficiency to C.)  At the moment this java software
> supports any table encoded according to the current FITS standard,
> and I'd like to be able to say the same about future versions.
>
> >            Although if somebody writes a class for a Rice (or PLIO or
> > HCOMPRESS) stream (gzip *is* supported) the problem can be nicely
> isolated.
>
> Yes, if they do.
>
> > (well javascript is not a serious language really suited to deal with
> binary
> > data, is it ?)
>
> I'm not a javascript/HTML5 expert, but there are increasingly complex
> web applications appearing for visualising and manipulating
> astronomical data within the browser.  Some of these do the data
> access on the server side, in which case use of CFITSIO can be used,
> but not all.  While investigating this, I found that the client-side
> javascript FITS I/O library fitsjs (https://github.com/astrojs/fitsjs)
> actually already offers partial support for tile-compressed images -
> though it looks like only the Rice algorithm is supported.
> I have CC'd Amit Kapadia, the author of that package, in case he
> wants to comment on the difficulties or otherwise of implementation
> (and doesn't already read this list).
>
> Mark
>
> --
> Mark Taylor   Astronomical Programmer   Physics, Bristol University, UK
> m.b.taylor at bris.ac.uk +44-117-9288776  http://www.star.bris.ac.uk/~mbt/
>
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/fitsbits/attachments/20150624/d7b1e789/attachment-0001.html>