[fitsbits] Start of "Foreign File Encapsulation" Public Comment Period

[LC's NoSpam Newsreading account]

I have been following the discussion on the FOREIGN extension for quite 
a bit, but found no time so far to communicate my views.

My opinions fall in three different categories :

a) procedural comments

   There was some confusion about the fact that a thing like the FOREIGN 
   extension should go in the convention registry, and how. They are
   probably due to the fact here we are dealing with a "conforming
   extension" and not a mere convention on keyword or column naming

This was first pointed out on Mon, 2 Oct 2006 by Thomas McGlynn

> The foreign file encapsulation doesn't really fit this framework.
> It describes an entirely new type of FITS HDU.  In the past such 
> a new type would require an amendment to the FITS standard.

(which is not really the case ... a new extension has just to be 
registered IN A SEPARATE REGISTRY, it is not necessarily part of the 
standard)

Also on Mon, 2 Oct 2006, William Pence wrote:

> The basic requirements for a convention to be included in the Registry
> are:
> 
>   1 It should conform to the FITS Standard.

  So I guess that until XTENSION='FOREIGN' is not registered in the
  registry of conforming extensions, it does not conform, and cannot
  be registered in the registry of conventions.

  AFAIK, things are dealt with in the IAU-FWG, so this is not a real
  issue

  I also agree with Bill when he says

> From my own perspective I would hope that the number of new extension 
> types that are created can be kept to a minimum because of the impact 
> that it has on existing software and the added confusion that it can 
> cause for FITS users.

  I will continue this sort of discussion on the IAUFWG list. For what
  the comments requested on FITSBITS about a new convention, they
  should properly deal ONLY with the fact the documentation is complete,
  well written, unambiguous, not redundant etc.

b) formal comments

So we are not really entitled to say anything about a particular 
implementation of a (local) convention. If the authors use it, they'd 
have good reasons to do so.

So the only really formal comments are those by Thomas McGlynn about 
some undefined acronyms, and about the fact that section 6 is part of an 
internal specification document and not relevant for the registry

> I'm not sure I understand what the acronyms MEF, FITS-MEF and EHU mean 
> in this proposal.  They never seem to be defined.

> Section 6 seems to be irrelevant to the proposal.


c) substantial comments
   
Now I'm a bit perplexed by a number of other items which have been 
pointed out by some participants to the discussion :

On Mon, 2 Oct 2006, Thomas McGlynn wrote:

> The encoding of directories (and symbolic links) is unclear.   
> Directories are not portable across systems. 

> The potential system dependencies of directory structures/file names 
> are not discussed. E.g., is c:\x\y\z\ a valid directory name?  How is 
> a absolute file path written on a Unix machine to be read on a Windows
[...]
> What happens when we go from a case sensitive to a case insensitive 
[...]

> I don't think Windows supports symbolic links either.

On the same date William Thompson wrote:

> 7.  I'm not sure I understand the point of encoding the user and group 
>    ID of the file.  Surely that's platform specific.

And finally on Tue, 3 Oct 2006, Peter Bunclark wrote:

> The file mode as a string ("rwx-rwx-rwx", bits not set given as "-").
>
> x ? Why would a file be executable? In the tradition of the
> omni-platform transportability of FITS files, it is unlikely [...]


As gut feeling I tend to AGREE with all above comments (and the rest by 
Thomas McGlynn), which can be grouped as "the proposal has clearly in 
mind Unix as target system".

For instance I could detail on Peter Bunclark's comment more than 
concerning with the execute bit being set, by the fact that a "file 
mode" made of 3 triplets rwxrwxrwx (NOT rwx-rwx-rwx !) for user, group 
and other is also NOT SYSTEM INDEPENDENT. If I recall well for instance 
VMS had 4 quadruplets RWED for user, group, world and system. I confess 
I ignore the Windows protection scheme.

These are technical comments. 

To these one might add that, if the origin and target systems are both 
Unix systems I fail to see the need to devise such a GENERAL convention 
in lieu of using a well established one like tar files (I may see the 
need of a simpler convention to append a single foreign file to a FITS 
file, like a PNG preview). This is a philosophical comment.

BUT WE ARE NOT ENTITLED TO MAKE SUCH COMMENTS WHEN DISCUSSING A 
CONVENTION TO GO IN THE CONVENTION REGISTRY.

We are not even entitled to ask the authors of such a convention to 
change anything (keyword names or contents) they already use. We can 
only say whether it is well documented.

Then, if NOAO has its own reasons to use such convention to move files 
around their site, they are in full right to do so and we ARE NOT 
ENTITLED to say anything like all the above comment items.


I believe it would be different if the FOREIGN extension had to be 
accepted as a STANDARD extension. ONOY in such case I believe we could 
question the need to "replace tar" and include a "directory structure" 
into a "FITS FOREIGN" file, and we SHALL question the conventions used
to store things like file names, case sensitivity, uid and gid, 
permissions in a system independent way, maybe requiring some limitation 
on character set, syntax etc.  I actually wonder if all these sort of 
things have not already been standardised in the way CDs are written by 
utilities like makeisofs / cdrecord.


A minor technicality, when Thomas McGlynn says
> I don't think Windows supports symbolic links either.

I do not believe that is true. Windows has at user level some kind of 
links providing pointers to a file, which resemble soft links, even VMS 
had, at system level, something like that (which resembled more hard 
links).

----------------------------------------------------------------------------
Lucio Chiappetti - INAF/IASF - via Bassini 15 - I-20133 Milano (Italy)
For more info : http://www.iasf-milano.inaf.it/~lucio/personal.html
----------------------------------------------------------------------------

-- 
----------------------------------------------------------------------
nospam at mi.iasf.cnr.it is a newsreading account used by more persons to
avoid unwanted spam. Any mail returning to this address will be rejected.
Users can disclose their e-mail address in the article if they wish so.