[fitsbits] JSONFITS, a suggestion for rendering FITS files in an Internet standards friendly format

[Christoph Deil]

> On 30 Sep 2015, at 02:09, Brian McConnell <bsmcconnell at gmail.com> wrote:
> 
> Hello,
> 
> Bill Pence suggested I post here. I'm working on a project to make
> FITS data more accessible to software developers who are not
> necessarily familiar with the format. Specifically, we're looking at
> ways to make SETI observational data available to third party
> developers who may be very knowledgeable about software engineering,
> digital signal processing, etc, but not field specific formats like
> this.
> 
> What I am working on is a utility that renders FITS files as JSON, a
> widely used interchange format (think XML without the bloat), with
> base64 encoded binary for payload data. The result is a file that is 7
> bit friendly, can be viewed in any text editor, and is trivial to
> parse. A typical JSONFITS file would look like:
> 
> [
> {"TARGNAME":"foo","TELESCOP":"arecibo","BINDATA":base64encodedblobofdatagoeshere},{"TARGNAME":"bar","TELESCOP":"arecibo","BINDATA":morebase64encodeddata}
> ]
> 
> Someone consuming this data can then do so very easily, as in the
> Python example:
> 
> blocks = json.loads(open('test.jsn','r').read())
> for b in blocks:
>  target_name = b.get('TARGNAME','')
>  telescope = b.get('TELESCOP','')
>  payload = base64.b64decode(b.get('BINDATA',''))
>  do_something_with(target_name, telescope, payload)
> 
> I should point out that I am not suggesting a new file format, but am
> thinking of this as an output filter for rendering FITS files in an
> internet friendly format.
> 
> Now you might ask won't base64 encoding bloat the file/download size?
> Yes, by about 4:3, but it turns out lossless compression largely
> undoes this, and as on the fly gzip compression is a built in feature
> in most web servers nowadays, this is basically a non issue (same
> thing for storage if you use a compressed volume). So you can have
> your cake (easy to parse, human readable files) and eat it too
> (similar overall footprint as binaries).
> 
> Well, I wanted to put this out there as a discussion item, and see if
> there's other work along these lines underway. My intent with the
> rendering utility is to make it available as a tiny Python library
> that people can use and build on.
> 
> Thanks for your time.
> 
> Brian McConnell <bsmcconnell at gmail.com>
> 
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits

Hi Brian,

I would like to use this.
So yes, please do share it!

Thanks,
Christoph