[fitsbits] 16-bit floats {External}

[Maren Purves]

We may have as much as a generation gap here (I think Paul Hirst goes
into that gap, but I don't know the other younger people personally)
as in a lot of other places.

A lot of us who have commented in this thread are past retirement age
and/or retired. I got out of this thread that there currently isn't an
IAU FITS WG.
Somebody has to take over, but I don't think that should happen
without the approval of important older people like Bill Pence, Steve
Allen, Rob Seaman, Jessica Mink (if still active)  and probably
several others (including my former office mate Malcolm Currie) I
forgot.

I work on the data acquisition end and currently have more work on that end.

Maren Purves
Head of Instrument and Telescope Software
East Asian Observatory / JCMT

On Sun, Jul 27, 2025 at 11:27 PM Francois-Xavier PINEAU via fitsbits
<fitsbits at listmgr.nrao.edu> wrote:
>
> Rust supports both f16 and f128 in its nightly (thus "experimental)" version:
> * https://doc.rust-lang.org/std/primitive.f16.html
> * https://doc.rust-lang.org/std/primitive.f128.html
> See also the RFC book
>   https://rust-lang.github.io/rfcs/3453-f16-and-f128.html
> pointing to some discussions
>   https://hackmd.io/@rust-lang-team/rkSQNidkp
> It should only be a matter of time before both become standard.
>
> But, I wonder if adding new BITPIX/TFORM values in HDUs
> already described in the standard (Primary, BINTABLE, ...) does fit
> with the FITS philosophy as it is described here:
>   https://fits.gsfc.nasa.gov/users_guide/users_guide/node7.html
>
> First paragraph:
>   "Any FITS reader should be able to cope with any FITS formatted file,
>    skipping over portions (extensions) and ignoring keywords that the reader
>    does not and need not understand"
>
> Knowing how to interpret BITPIX and TFORM is mandatory to compute the byte size
> of the data part of a HDU, and thus to be able to skip it.
> True that the BITPIX description contains "[t]he absolute value is used in
> computing the sizes of data structures", but there is a list of "valid" values
> and I would expect a lot of implementations to raise an error when encountering
> an undefined BITPIX value.
> In addition, there is no way to know the byte size of an unknown TFORM value.
>
> Then, true:
>   "Changes in the FITS rules may add new structures that old software cannot
>    handle. Revised software will be required for new standard extensions [...]"
>
> but, followed by:
>   "As far as is possible, however, FITS should be expanded in such a way that
>    the old software will still be able to process those parts of the file which
>    it is capable of handling.
>    In such a case, software should not fail or give incorrect results when
>    confronted with the new extension or conventions; it should simply ignore
>    them and continue to process those parts of the file that it can
>    understand."
>
> "As far as is possible" is open to interpretation, but "new structures" and
> "required for new standard extensions" does not seem to be compatible (to me)
> with *current standard extension* and *new values of already defined keywords*.
> Does the "FITS philosophy" cover the case of breaking changes in existing extensions?
> Should it be reworded or expanded?
>
> Quite clearly, I expect most (all/a lot of?) software to fail trying to read a FITS
> file with previously not defined BITPIX or TFORM values (as mentioned
> by Preben Grosboel).
>
> I do understand that defining new extensions to simply add new BITPIX/TFORM
> values seems excessive.
>
>
> Also copying with the question of the real need for f16 (and/or f128) in FITS,
> an alternative could be to start by introducing two new FITS conventions:
> * one generic convention describing how to overwrite existing keywords values
>   (OVERWRITE keyword?)
> * one specific "half-float" convention introducing BITPIX=-16 / TFORM=H
>   and using the generic "overwrite" convention to hide those values to
>   unmaintained software.
>
> An image containing half floats could be described as a flat byte array in
> "regular" FITS (nothing interesting to be read for "half-float" unaware
> readers), and decoded by *overwrite + half-float* aware readers.
>
> Similarly, in a BINTABLE, a half float column could be a 2B column for
> regular readers and a f16 for *overwrite + half-float* aware readers.
>
> Time will let know is those conventions are worth being put in the standard
> or not (or if the introduction of f16 in the standard is worth breaking
> existing software).
>
>
> Maybe defining new conventions here is also excessive.
> If so, should the FITS philosophy page be (slightly) re-worded/extended?
>
> If I remember correctly, long keywords have not been accepted because:
> * it would have break existing software
> * the HIERARCH convention already allows to define long keywords
> But, if breaking changes like new BITPIX/TFORM values are accepted,
> what about other breaking changes: long keywords, UTF-8 values, ... ?
>
>
> I don't have any firm convictions; I'm simply trying to better understand
> the FITS philosophy and to explore the various possibilities.
>
>
> François-Xavier Pineau
>
> Le 23/07/2025 à 22:40, Mohammad Akhlaghi via fitsbits a écrit :
>
> As mentioned before, it is also available in C:
>
> https://www.gnu.org/software/c-intro-and-ref/manual/html_node/Floating-Representations.html
>
> Cheers,
> Mohammad
>
>
>
> On July 23, 2025 10:22:12 PM GMT+02:00, "Barrett, Paul via fitsbits" <fitsbits at listmgr.nrao.edu> wrote:
>>
>> These formats are available in the Julia programming language. Adding them to FitsFiles.jl should take minimal effort, say a few days at most.
>>
>>  -- Paul
>>
>>
>> On Wed, Jul 23, 2025, 16:13 Lucio Chiappetti via fitsbits <fitsbits at listmgr.nrao.edu> wrote:
>>>
>>> On Wed, 23 Jul 2025, Barrett, Paul via fitsbits wrote:
>>>
>>> > Yes, definitely. I have been advocating for half-precision (16-bit)
>>> > floating point for several years now for radio astronomy. In addition,
>>> > 128-bit floats
>>>
>>> Are these formats supported by any major programming language ?
>>>
>>> Is it worth supporting "exotic" formats, which might be suitable for some
>>> niche application, when most data producers often use improperly 64-bit
>>> all the times, even when overshoot ?
>>>
>>> Historically FITS went the other way round (concentrating on 16 and 32,
>>> later 64, when mainframres with 36 or 60 bits were around).
>>>
>>> _______________________________________________
>>> fitsbits mailing list
>>> fitsbits at listmgr.nrao.edu
>>> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>
>
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.nrao.edu
> https://listmgr.nrao.edu/mailman/listinfo/fitsbits