[fitsbits] 16-bit floats {External}

[Mohammad Akhlaghi]

Hi Rob,

Indeed, the tiled compression is very elegant and useful in many 
contexts and will indeed decrease the file size without loss of 
information if the extra 16 bits of a 32-bit floating point image are 
intentionally set to zero. I am currently advocating tiled compression 
for the data model of our ARRAKIHS mission (by ESA). I really appreciate 
the wonderful work that has gone into into it and how useful it is!

The main problem with using tiled compression to replace the need for a 
16-bit floating point is the extra 16 bits. By default, they will get 
filled with floating-point errors of the compiler/CPU. So to have a good 
lossless compression, the writing library will have to ensure to set 
them to zero before using the tiled compression.

For long term archival, this extra effort makes sense and is reasonable. 
But when you just want to write a temporary table as part of a larger 
pipeline and read it in a later phase, all the overhead (of making and 
then unpacking) can be non-negligible.

So if there is a native 16-bit float type, it would help both in terms 
of intermediate file products (storage), in-memory operations (less RAM 
while the program is running), all without any overhead.

Cheers,
Mohammad

On 7/23/25 4:30 PM, Seaman, Robert Lewis - (rseaman) via fitsbits wrote:
> Hi all,
> 
> Any such changes should be coordinated with the recent suggestion of adding JPEGXL support to tile compression.
> 
> If the IAU FITS WG is going through a period of rebuilding, the responsibility for maintaining FITS falls to Commission B2.
> 
> We looked into support for 16-bit floats when working on FPACK and tile compression. As I recall, it may not be as standardized as 32-bit and 64-bit IEEE floats. There were also some video (?) standards that used real-number representations that were fixed-point, not floating-point.
> 
> 128-bit formats may need to coexist with complex number representations. The motivations for both shorter than usual or longer than usual floating-point representations in FITS should flow from explicit science and engineering use cases.
> 
> Tiled 32-bit integer compression is actually quite elegant, and the noise-sensitive floating-point tile compression quite powerful, see Paper I and Paper II linked at https://heasarc.gsfc.nasa.gov/fitsio/fpack/. There are also a number of interesting issues associated with efficient table compression, such as the benefit of transposing tables and shuffling the bytes. Simply adding another data type won’t address all the issues.
> 
> I am unaware of any coherent benchmarks comparing the speed of using 16-bit floats versus FPACK-style floating point compression. This should also be a prerequisite. Are 16-bit floats widely used in other scientific disciplines?
> 
> Rob Seaman
> Lunar and Planetary Laboratory
> 
> 
> 
> 
> 
> On 7/23/25, 6:55 AM, "fitsbits" wrote:
> 
> External Email
> 
> On Wed 2025-07-23T13:34:54+0100 Thomas Robitaille via fitsbits hath writ:
>> I am aware of some modern projects that would benefit from having 16-bit
>> floats, since they consider it to be sufficient in precision to store very
>> large datasets, and using 16-bit floats would perform a lot better than
>> using compression on 32-bit floats for example, and 16-bit floats would
>> allow a larger dynamic range than using 16-bit ints with BSCALE/BZERO.
> 
> This is not a breaking change, and with little more discussion than we
> already see today an informal agreement cold quickly go into use and
> then be adopted.
> I think the harder part about making this change is establishing
> the auspices under which the IAU FITS working group currently acts.
> 
> --
> Steve Allen <sla at ucolick.org> WGS-84 (GPS)
> UCO/Lick Observatory--ISB 260 Natural Sciences II, Room 165 Lat +36.99855
> 1156 High Street Voice: +1 831 459 3046 Lng -122.06015
> Santa Cruz, CA 95064 https://www.ucolick.org/~sla/<https://www.ucolick.org/~sla> Hgt +250 m
> 
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits<https://listmgr.nrao.edu/mailman/listinfo/fitsbits>
> 
> 
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits