[fitsbits] Start of "Foreign File Encapsulation" Public Comment Period

[Doug Tody]

Hi Tom -

These are all good comments, but I think we need to be clear on
what the registry of FITS conventions is intended to be used for.
I thought it was to be used (at least in part) to document existing
FITS conventions - NOT standards - which are already in use.  In the
case of the foreign file convention we are not trying to design a
new FITS standard, but merely document and register something which
has been in use for some years.  There are numerous other existing
conventions of this type.  If this is our goal then it is inappropriate
to consider making major changes to what is an existing convention.
If instead our goal is to design a new FITS standard, that is a
very different thing and we might do things very differently this
time around.

Also - FITS defines a general extension mechanism which makes it
possible to, well, define extensions to the standard.  Isn't that
what a convention is?

 	- Doug



On Mon, 2 Oct 2006, Thomas McGlynn wrote:

> I'm not entirely clear on how the FITS conventions registry is
> supposed to work.   My original concept was that this was
> to be a way in which FITS data is used in ways
> that are fully compliant with existing FITS standards but which have
> additional semantics.  The registry was to
> expose these ideas so that others could adopt them into
> their own software and recognize them when we saw them.
>
> The foreign file encapsulation doesn't really fit this framework.
> It describes an entirely new type of FITS HDU.  In the past such
> a new type would require an amendment to the FITS standard.
> So in this case is the registry functioning as a kind of second
> tier of 'approval' for a new FITS standard?  I'm not sure I
> think this is appropriate or
> at least I'd like to understand how I, as a writer of FITS
> software am expected to respond to this proposal.  If this
> standard is approved should I modify my software to understand
> XTENSION=FOREIGN?  Am I forbidden (strongly discouraged?)
> from using XTENSION=FOREIGN for my own FITS extension?  Does
> this ever move on to become part of the formal FITS standard, or are
> these conventions parallel to the standard?
>
> ---
>
> All that said,
> I think that there are two very distinct elements to this and it might
> well do better split into two conventions.
>
> The first would be the definition of the XTENSION=FOREIGN HDU.
> This convention describes the mapping between an HDU and a single file (or possibly a
> directory).
>
> The second would be the convention on how HDU's are arranged and keywords set to create
> the FITS equivalent of a TAR file.
>
> These are the major comments,  I've a number of smaller issues described below.
> There are written from the viewpoint of trying to see how I might use this
> convention in a general framework.  If that's not what's intended many of
> them may not apply.
>
> 	Regards,
> 	Tom McGlynn
>
>
>
> ---
> There is discussion of a table of contents in the primary header unit.
> If something other than comments is intended here, the format of this
> needs to be specified.  If only a set of comments was intended, no discussion
> is needed.  What happens when more than one file group is included in
> a given FITS file?
>
> ----
>
> The encoding of directories (and symbolic links) is unclear.   Directories are not portable
> across systems.  Are directories intended only to be markers
> (e.g., HDUs with PCOUNT=0) or was some data intended to be encoded here.
>
> ----
>
> The potential system dependencies of directory structures/file names are not discussed.
> E.g., is c:\x\y\z\ a valid directory name?  How is a absolute file path written on a
> Unix machine to be read on a Windows machine if there are multiple
> top level directories (e.g., C: and D:)? What happens when we
> go from a case sensitive to a case insensitive architecture and there are
> files that differ only in case?  [Doubtless more pain here...]
>
> I don't think Windows supports symbolic links either.
>
> ----
>
> Are both absolute and relative directory paths supported?
>
> ---
>
> I'm not sure I understand what the acronyms MEF, FITS-MEF and EHU mean in this
> proposal.  They never seem to be defined.
>
> ---
>
> Is it illegal to encode FITS data, or does the current FITS software simply
> choose not to do so for convenience?
>
> ---
>
> FG_GROUP
> Can data from multiple groups be intermingled?  So that we get an HDU
> from one group followed by data from another followed by another file from the first
> group?  The proposal implies that multiple groups can be part of a single file.
> Does a group need to be contiguous?
>
> ---
> FG_FNAME
> Does/can this include the directory of the file?  Does it inherit the
> directory from the last directory seen?  If a directory do we build up
> directories with a sequence of them? [E.g., if we get the following files
>    directory=x directory=y, directory=z, file=a do
> we get a file x/y/z/a or wo we have ./x ./y ./z and ./a?]
>
> ---
> FG_FSIZE
> For FSIZE What is meant by the 'data' portion of a file?  This is different from
> PCOUNT how?
>
> ---
> FG_FMODE
> The meaning should be given more explicitly (e.g., does owner or world come first)?
> What happens on systems where this is not present? (Also FUGRP)
>
> ---
>
> It's is unclear which keywords are optional and which are required.
> If any keywords are optional their defaults are not specified.  (FG_MTYPE
> is indicated to be optional.  Does that mean everything else is
> required?)
>
> ---
>
> Section 6 seems to be irrelevant to the proposal.
>
>
> _______________________________________________
> fitsbits mailing list
> fitsbits at listmgr.cv.nrao.edu
> http://listmgr.cv.nrao.edu/mailman/listinfo/fitsbits
>