<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi all,</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I agree with Bill and Arnold (as ever and always) about the meaning of once-FITS, always-FITS, but feel the need to point out that UTF-8 is a variable-length encoding. What is the precise proposal here? For instance, it ought to be possible to add variable-length
encodings to variable-length arrays on the heap,</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
though it would complicate the overall length calculations. But it seems a waste of time to try to squeeze UTF-8 (in all its glory) into ASCII FITS header records. This is another good reason to de-emphasize or deprecate ASCII FITS headers for header-style
metadata in a bintable.</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Rob Seaman</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Catalina Sky Survey</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Lunar and Planetary Laboratory</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
---</div>
<div style="direction: ltr; font-family: Aptos, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="mail-editor-reference-message-container">
<div class="ms-outlook-mobile-reference-message skipProofing" style="direction: ltr;">
</div>
<div class="ms-outlook-mobile-reference-message skipProofing">On 4/6/26, 2:55 AM, "fitsbits" wrote:</div>
<div class="ms-outlook-mobile-reference-message skipProofing"><br>
Thank you Bill.<br>
<br>
To confirm, solution 1 proposed by FX, changing the interpretation<br>
of TFORM=A to be UTF-8 rather than ASCII, would be fully consistent<br>
with this "once FITS always FITS" rule. Character content as<br>
currently defined would remain valid FITS, with the same interpretation<br>
as now, since ASCII encoding is identical to UTF-8 encoding for<br>
any currently legal FITS character (characters 0x20 to 0x7E).<br>
<br>
Mark<br>
<br>
On Sat, 4 Apr 2026, William Pence via fitsbits wrote:<br>
<br>
> Roughly translated, Francois wrote: “it would have seemed important to keep the distinction between pure ascii and utf-8, because of the FITS logic to ensure that changes in the standard should not affect existing applications would be violated”.<br>
><br>
> Just to be clear, there has never been a requirement that changes to the definition of the FITS format must not cause problems for existing software applications. The actual requirement, as stated in section 3.7 of the FITS Standard is that “Any structure
that is a valid FITS structure shall remain a valid FITS structure at all future times” (often referred to as the “once FITS, always FITS” rule).<br>
><br>
> Bill<br>
><br>
><br>
> > On Apr 3, 2026, at 11:16 PM, Francois Ochsenbein via fitsbits <fitsbits@listmgr.nrao.edu> wrote:<br>
> ><br>
> > <br>
> > Bonjour FX,<br>
> ><br>
> > Apparemment le « current draft of VOTable 1.6 » n'existe pas ?<br>
> > Personnellement il m'aurait semblé important de conserver la<br>
> > distinction entre pur ascii et utf-8, car la logique de FITS de<br>
> > garantir que les changements dans le standard ne doivent pas affecter<br>
> > les applications existantes serait violée… Mais je serais curieux de<br>
> > savoir en quoi le HATS a besoin d'UTF-8 ?<br>
> ><br>
> > En te souhaitant une belle fin de semaine, et peut-être à bientôt ?<br>
> ><br>
> > Bien amicalement, François<br>
> ><br>
> > ==> Le jeudi 2026-03-26 à 14:43+0100,<br>
> > Francois-Xavier PINEAU via fitsbits <fitsbits@listmgr.nrao.edu> a<br>
> > écrit:<br>
> ><br>
> >> Dear fitsbits,<br>
> >><br>
> >><br>
> >> # Background<br>
> >><br>
> >> VOTable (v1.5) is closely compatible with the FITS Binary Table format:<br>
> >> <a href="https://www.ivoa.net/documents/VOTable/20250116/REC-VOTable-1.5.html#tth_sEc2.3" data-outlook-id="54ebee02-4e2e-49b5-ba8e-dbe04640edf0">
https://www.ivoa.net/documents/VOTable/20250116/REC-VOTable-1.5.html#tth_sEc2.3</a><br>
> >><br>
> >> In the current draft of VOTable 1.6<br>
> >> <a href="https://github.com/ivoa-std/VOTable/releases/download/auto-pdf-preview/VOTable-draft.pdf" data-outlook-id="3e009537-29ed-4ca8-b049-38bcbf9c1082">
https://github.com/ivoa-std/VOTable/releases/download/auto-pdf-preview/VOTable-draft.pdf</a><br>
> >> ,<br>
> >> UTF-8 strings replace the previous ASCII-only strings.<br>
> >><br>
> >> If FITS cannot store UTF-8, *lossless round-trip conversion from<br>
> >> VOTable to FITS will no longer be possible*.<br>
> >> Some limitations already exist (e.g., unsigned integer logical types),<br>
> >> but UTF-8 seems more critical.<br>
> >><br>
> >> Personal use cases include the usage of HEALPix sorted and indexed<br>
> >> BINTABLES to build on-the-fly HATS products<br>
> >> or intermediary HiPS catalogue representations from VizieR data (will<br>
> >> contains more and more UTF-8).<br>
> >> * HATS: <a href="https://www.ivoa.net/documents/Notes/HATS" data-outlook-id="7cda9e70-813b-409a-90d5-b11b33937479">
https://www.ivoa.net/documents/Notes/HATS/</a><br>
> >> * HIPS catalogue: <a href="https://www.ivoa.net/documents/HiPS" data-outlook-id="b016026e-6dd1-443f-8721-383fc83c0ab1">
https://www.ivoa.net/documents/HiPS/</a><br>
> >> * VizieR: <a href="https://vizier.u-strasbg.fr" data-outlook-id="3c4c2d9c-5ad2-4fec-a82e-be36ebca9e7d">
https://vizier.u-strasbg.fr/</a><br>
> >><br>
> >> # Possible Solutions<br>
> >><br>
> >> ## 1. Use UTF-8 in existing `TFORMn=rA`<br>
> >><br>
> >> Like in VOTAble 1.6, interpret `r` as bytes instead of characters.<br>
> >> May break truncation operations (TDIPS) if a multi-byte UTF-8<br>
> >> character is split.<br>
> >><br>
> >> ## 2. Logical type "UTF-8" backed by a byte array<br>
> >><br>
> >> TFORMn = rB<br>
> >> TLOGTn = 'UTF-8' / LOGT stands for LOGical Type<br>
> >><br>
> >> Unaware readers see a byte array; UTF-8 aware readers interpret it as<br>
> >> a string.<br>
> >> Introduces two string types in FITS (ASCII and UTF-8).<br>
> >><br>
> >> ## 3. New TFORM type (e.g., `TFORMn=rU`)<br>
> >><br>
> >> Definite breakage for current readers.<br>
> >><br>
> >><br>
> >> # Existing Implementations<br>
> >><br>
> >> * TOPCAT/STILTS (Java): Prototype supports Solutions 1 and 2 for<br>
> >> read/write (private communication with Mark Taylor).<br>
> >> * fitstable (Rust): Supports Solutions 1 and 2 for reading<br>
> >> (<a href="https://github.com/cds-astro/cds-fitstable-rust" data-outlook-id="6d73480d-9e22-4678-a4f9-dbd263b5bb5d">https://github.com/cds-astro/cds-fitstable-rust</a>).<br>
> >> * VizieR: Appears to provide UTF-8 in TFORMn=rA columns (Solution 1).<br>
> >> * ??<br>
> >><br>
> >><br>
> >> # Feedback Requested<br>
> >><br>
> >> I am curious about:<br>
> >> * other possible approaches<br>
> >> * fitsbits opinions on the most practical solution<br>
> >> * other people interested in having UTF-8 in BINTABLE columns<br>
> >><br>
> >> Currently, Solution 1 seems the simplest and Solution 2 the safest,<br>
> >> but I welcome constructive comments and experience from the community.<br>
> >><br>
> >> Best regards,<br>
> >><br>
> ><br>
> ><br>
> > --<br>
> > ======================================================================<br>
> > Francois Ochsenbein@free.fr --- 67380 Lingolsheim<br>
> > ======================================================================<br>
> ><br>
> > _______________________________________________<br>
> > fitsbits mailing list<br>
> > fitsbits@listmgr.nrao.edu<br>
> > <a href="https://listmgr.nrao.edu/mailman/listinfo/fitsbits" data-outlook-id="ff84d88a-2bd2-4792-b153-c5311bd64d92">
https://listmgr.nrao.edu/mailman/listinfo/fitsbits</a><br>
><br>
><br>
> _______________________________________________<br>
> fitsbits mailing list<br>
> fitsbits@listmgr.nrao.edu<br>
> <a href="https://listmgr.nrao.edu/mailman/listinfo/fitsbits" data-outlook-id="9168980d-14f4-4051-8622-c63c021cd815">
https://listmgr.nrao.edu/mailman/listinfo/fitsbits</a><br>
><br>
<br>
--<br>
Mark Taylor Astronomical Programmer Physics, Bristol University, UK<br>
m.b.taylor@bristol.ac.uk <a href="https://www.star.bristol.ac.uk/mbt" data-outlook-id="52475d0b-c88e-4926-851d-88e83f592a6d">
https://www.star.bristol.ac.uk/mbt/</a></div>
</div>
</body>
</html>