[fitsbits] 16-bit floats {External}

[Malcolm J. Currie]

Being (allegedly) retired and having little contact with the
astronomical programming world post COVID, it's no surprise that
I too was unaware of 16-bit floats before this discussion.

My sympathies are with Walter's view, as I'm inclined towards regarding
FITS as an interchange format rather than a processing format.  My bias
is partly due to having our own flexible and extensible format in
Starlink.  I'm wary of additions to the standard that have a quite
limited usage solely for FITS as a working format.

I agree with the points and questions made by Rob, especially how widely
would these two additions be used and evidence of their benefits.  A
document for discussion covering these points, how the new data types
would be implemented, and the implications for existing software
packages should be presented.  In many packages the float16 FITS data
would presumably have to be converted to another data type, or say it's
not supported.

Starlink's data system is now built on HDF5 (rather than the in-house
HDS that preceded it).  A quick search turned up an RFC to add 16-bit
floats to HDF5, and a long user discussion of the RFC, but I've yet to
find HDF's justification for the introduction of float16, or whether it
has been implemented.

Wikipedia, that trusty source of knowledge and wisdom,
(https://en.wikipedia.org/wiki/Half-precision_floating-point_format)
relates that the concept has been around as early as 1982.  Also I'd
forgotten IEEE 754 had a binary16, presumably because there wasn't a
perceived need and it wasn't in the floating-point addition to FITS.
We already had BSCALE/BZERO to increase the dynamic range.  The
Wikipedia entry does give some reasons for using 16-bit floats, such as
machine learning, and lists programming languages that support it, but
I'd certainly like to hear more on potential benefits in astronomy.

Lucio:
> That is true, and it was what I referred to as the early times when one
> could write a reader of FITS magtapes on 36-bit or 60-bit mainframes. Or
> different endianness.

Hence the strange, to a young audience, 2880-byte logical record length.
That accompanies the restrictive 80-byte headers that must look so
antiquated to postdocs and students.  Are they doing their own thing
rather than join in the FITS discussion?  Most of the participants in
this thread are of, let's say, a certain vintage.

Lucio:
> Also I seem to remember in the past there was a rule, or at least a
> practice, that a new feature has tp be supported by two independent
> existing implementattions.

Indeed.

>> And finally, finally, discussions like this and about JPEGXL demonstrate
>> that we need to revive the IAU FITS WG.
>
> And that's even more true than all the rest, specially for a delicate item
> like basic data formats.

Yes but who would serve on it?  There needs to be a mix of decades of
FITS experience, combined with a new generation who work with
cutting-edge data and state-of the-art tools or who are designing
upcoming major data-generating projects.

Malcolm