[fitsbits] reopening of Public Comment Period on the compression conventions

[William Thompson]

Since we've now moved into the petabyte era in many areas of astronomy, and are 
closing on the exabyte era, I don't think this restriction to only lossless 
compression is supportable anymore.  It's the responsibility of the data 
provider to make sure that the data are presented with sufficient quality to 
achieve the scientific requirements.  As I pointed out before, many space 
missions already use lossy compression schemes to keep within their telemetry 
limitations.

Bill Thompson


On 01/14/16 15:32, Demitri Muna wrote:
> Hi,
>
> I don’t have any opinions on compression algorithms, but I want to note that I’m
> opposed to the FITS format supporting any lossy compression as part of the
> format. FITS should be archival, and I think that any lossy representation of
> data (e.g. a thumbnail) should be created outside of the file. Lossy algorithms
> can improve dramatically over time with the increase of CPU speed (see the ~50%
> space improvement from H.264 to H.265 for the same quality). Thumbnails are
> extremely cheap to create and cache and become cheaper to do so over time with
> faster I/O and CPU. It’s common in archives to have jpg files next to FITS files
> for this reason - I don’t see a benefit to building this into the format. Keep
> things light.
>
> Cheers,
> Demitri
>
> _________________________________________
> Demitri Muna
>
> Department of Astronomy
> Le Ohio State University
>
> Home page: http://muna.com
>
> My Projects:
> http://nightlightapp.io
> http://trillianverse.org
> http://scicoder.org
>
>
>
>
>
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>

-- 
William Thompson
NASA Goddard Space Flight Center
Code 671
Greenbelt, MD  20771
USA

301-286-2040
William.T.Thompson at nasa.gov