[fitsbits] Maximum number of extensions

[Richard West]

"Clive Page" <cgp at nospam.le.ac.uk> wrote in message
news:bacplb$67psn at harrier.le.ac.uk...
>
> That flexibility is good, I guess, but perhaps it would be better if you
> didn't tell everyone. :-)  I really think the use of huge numbers of HDUs
> ought to be discouraged.  There is no index to them, so the only way a
> reading program has of finding a specific HDU is to scan the header lines
> of every preceding HDU.  Similarly the only way of finding the total
> number of HDUs in a FITS file is to move to the "next" HDU and find that,
> after a few thousand or whatever, it doesn't exist. You could, I suppose
> design and insert your own index near the front, but of course if that's a
> table it can't be in the primary HDU.  Wouldn't it be better to just have
> lots of files, each with one or two HDUs?   Modern file systems cope so
> much better with data access, don't they?
>

You might very well think so, however a naive solution to my problem would
yield of order 10^8 individual files, each of which would be updated once a
day. Some initial tests suggest that the file system becomes a major
bottle-neck if used in this way (ie. it would take ~3 days just to open and
close the files!).

So I'm considering some (hopefully) less naive solutions :-)