[fitsbits] 16-bit floats {External}
Francois-Xavier PINEAU
francois-xavier.pineau at astro.unistra.fr
Mon Jul 28 05:56:36 EDT 2025
(With unknown TFORM, one can still skip the full BINTABLE HDU or
still read columns located before the unknown TFORM column.
One could also skip unknown TFORM columns, but only if the bintable
contains a single unknown TFORM type.
That said, I do not expect software to have implemented such
possibilities,
they will probably just raise an error when encountering an unknown TFORM.
).
Le 28/07/2025 à 11:26, Francois-Xavier PINEAU a écrit :
>
> Rust supports both f16 and f128 in its nightly (thus "experimental)"
> version:
> * https://doc.rust-lang.org/std/primitive.f16.html
> * https://doc.rust-lang.org/std/primitive.f128.html
> See also the RFC book
> https://rust-lang.github.io/rfcs/3453-f16-and-f128.html
> pointing to some discussions
> https://hackmd.io/@rust-lang-team/rkSQNidkp
> It should only be a matter of time before both become standard.
>
> But, I wonder if adding new BITPIX/TFORM values in HDUs
> already described in the standard (Primary, BINTABLE, ...) does fit
> with the FITS philosophy as it is described here:
> https://fits.gsfc.nasa.gov/users_guide/users_guide/node7.html
>
> First paragraph:
> "Any FITS reader should be able to cope with any FITS formatted file,
> skipping over portions (extensions) and ignoring keywords that the
> reader
> does not and need not understand"
>
> Knowing how to interpret BITPIX and TFORM is mandatory to compute the
> byte size
> of the data part of a HDU, and thus to be able to skip it.
> True that the BITPIX description contains "[t]he absolute value is
> used in
> computing the sizes of data structures", but there is a list of
> "valid" values
> and I would expect a lot of implementations to raise an error when
> encountering
> an undefined BITPIX value.
> In addition, there is no way to know the byte size of an unknown TFORM
> value.
>
> Then, true:
> "Changes in the FITS rules may add new structures that old software
> cannot
> handle. Revised software will be required for new standard
> extensions [...]"
>
> but, followed by:
> "As far as is possible, however, FITS should be expanded in such a
> way that
> the old software will still be able to process those parts of the
> file which
> it is capable of handling.
> In such a case, software should not fail or give incorrect results
> when
> confronted with the new extension or conventions; it should simply
> ignore
> them and continue to process those parts of the file that it can
> understand."
>
> "As far as is possible" is open to interpretation, but "new
> structures" and
> "required for new standard extensions" does not seem to be compatible
> (to me)
> with *current standard extension* and *new values of already defined
> keywords*.
> Does the "FITS philosophy" cover the case of breaking changes in
> existing extensions?
> Should it be reworded or expanded?
>
> Quite clearly, I expect most (all/a lot of?) software to fail trying
> to read a FITS
> file with previously not defined BITPIX or TFORM values (as mentioned
> by Preben Grosboel).
>
> I do understand that defining new extensions to simply add new
> BITPIX/TFORM
> values seems excessive.
>
>
> Also copying with the question of the real need for f16 (and/or f128)
> in FITS,
> an alternative could be to start by introducing two new FITS conventions:
> * one generic convention describing how to overwrite existing keywords
> values
> (OVERWRITE keyword?)
> * one specific "half-float" convention introducing BITPIX=-16 / TFORM=H
> and using the generic "overwrite" convention to hide those values to
> unmaintained software.
>
> An image containing half floats could be described as a flat byte
> array in
> "regular" FITS (nothing interesting to be read for "half-float" unaware
> readers), and decoded by *overwrite + half-float* aware readers.
>
> Similarly, in a BINTABLE, a half float column could be a 2B column for
> regular readers and a f16 for *overwrite + half-float* aware readers.
>
> Time will let know is those conventions are worth being put in the
> standard
> or not (or if the introduction of f16 in the standard is worth breaking
> existing software).
>
>
> Maybe defining new conventions here is also excessive.
> If so, should the FITS philosophy page be (slightly) re-worded/extended?
>
> If I remember correctly, long keywords have not been accepted because:
> * it would have break existing software
> * the HIERARCH convention already allows to define long keywords
> But, if breaking changes like new BITPIX/TFORM values are accepted,
> what about other breaking changes: long keywords, UTF-8 values, ... ?
>
>
> I don't have any firm convictions; I'm simply trying to better understand
> the FITS philosophy and to explore the various possibilities.
>
>
> François-Xavier Pineau
>
> Le 23/07/2025 à 22:40, Mohammad Akhlaghi via fitsbits a écrit :
>> As mentioned before, it is also available in C:
>>
>> https://www.gnu.org/software/c-intro-and-ref/manual/html_node/Floating-Representations.html
>>
>> Cheers,
>> Mohammad
>>
>>
>>
>> On July 23, 2025 10:22:12 PM GMT+02:00, "Barrett, Paul via fitsbits"
>> <fitsbits at listmgr.nrao.edu> wrote:
>>
>> These formats are available in the Julia programming language.
>> Adding them to FitsFiles.jl should take minimal effort, say a few
>> days at most.
>>
>> -- Paul
>>
>>
>> On Wed, Jul 23, 2025, 16:13 Lucio Chiappetti via fitsbits
>> <fitsbits at listmgr.nrao.edu> wrote:
>>
>> On Wed, 23 Jul 2025, Barrett, Paul via fitsbits wrote:
>>
>> > Yes, definitely. I have been advocating for half-precision
>> (16-bit)
>> > floating point for several years now for radio astronomy.
>> In addition,
>> > 128-bit floats
>>
>> Are these formats supported by any major programming language ?
>>
>> Is it worth supporting "exotic" formats, which might be
>> suitable for some
>> niche application, when most data producers often use
>> improperly 64-bit
>> all the times, even when overshoot ?
>>
>> Historically FITS went the other way round (concentrating on
>> 16 and 32,
>> later 64, when mainframres with 36 or 60 bits were around).
>>
>> _______________________________________________
>> fitsbits mailing list
>> fitsbits at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
>>
>>
>> _______________________________________________
>> fitsbits mailing list
>> fitsbits at listmgr.nrao.edu
>> https://listmgr.nrao.edu/mailman/listinfo/fitsbits
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/fitsbits/attachments/20250728/2fd5dbd4/attachment.html>
More information about the fitsbits
mailing list