[fitsbits] Unable to make a large Fits Table

[Oisin Creaner]

Hello,
I am a Ph.D. student from Ireland working on a project that data mines the
SDSS catalogue.   I was advised to ask here for help by Peter Kretschmar of
ESA who is a colleague of my supervisor.  I sent a similar email in at about
the same time I signed up for this list, about a week ago, but since I
recieved my confirmation email after this point, I am unsure if it would
have been recieved.   If anyone has recieved a double post from me, I
apologise.

I am having trouble with a C++ program that uses the cfitsio library .  The
program is intended to write a large amount of data from an array in memory
into a .fits table.  However, I get an error when I attempt to write an
array of more than 14040 "double" numbers to the file.

If I attempt to write between 14041 and 14400 doubles, I get this error.
*FITSIO status = 107: tried to move past end of file
Error reading data buffer from file:* *
/mnt/Share/test_harness_out_fits.fit
Error while closing HDU number 1 (ffchdu).*

While if I write 14401 or over I get this one
*FITSIO status = 252: 1st key not SIMPLE or XTENSION
Error reading data buffer from file:* *
/mnt/Share/test_harness_out_fits.fit
Extension doesn't start with SIMPLE or XTENSION keyword. (ffrhdu)* *
@B
Error while closing HDU number 1 (ffchdu).*

I can't seem to find any documentation indicating such a limit on the size
of fits tables (indeed, what I've found seems to suggest I'm barely
scratching the surface.)

I have included a stripped down version of my code below which includes the
critical functions and which returns the same errors and at the same points
in the code as the "live" code does.

I am running scientific Linux 4.5 and using Eclipse Europa as my development
environment.

I hope this is a relatively simple fix and you can help me.
Many thanks in advance,
Oisin Creaner

using namespace std;
// Required header files
#include <stdio.h>
#include <stdlib.h>

//required for FITS file operations
#include <fitsio.h>

//includes the mathemathical library
#include <math.h>

//This function creates a fits file
int make_out_fits ( fitsfile **p_fp_out_fits, char *p_file_name);

//This function writes the array into a fits file
int write_output_fits(    fitsfile **p_fp_out_fits,
                        double **p_output_array,
                        long int *p_total_targets);

int main() {
    //error handling routine
    int error = 0;

    //Test data: normally calculated based on inputs
    //works at up to 14040// gives error 107 from 14041-14400//gives error
252 beyond14401;
    long int out_num_rows = 14000;
    long int *p_out_num_rows = &out_num_rows;

    //index for seeding array
    int output_index = 0;

    //Fitsio error handling
    int status = 0;

    //creates a pointer to a file to store the output filename (150
characters allowed)
    char *p_out_file_name= (char *) malloc (150 * sizeof(char));

    //creates a pointer for the output fits file
    fitsfile *fp_out_fits = {NULL};
    fitsfile **p_fp_out_fits = &fp_out_fits;

    //creates an array to write to a file
    double *output_array= (double *) malloc (out_num_rows *(sizeof
(double)));
    double **p_output_array  = (double **) malloc (sizeof (double));
    p_output_array  = &output_array;

    //output file name
    p_out_file_name = "/mnt/Share/test_harness_out_fits_small.fit";

    //seeds the array //in the real program, the array is generated from
inputs
    for (output_index = 0; output_index < out_num_rows; output_index ++)
    {
        *(output_array+output_index) = output_index;
        printf("%lf \n", *(output_array+output_index));
    }

    //and makes the file
    error = make_out_fits (p_fp_out_fits, p_out_file_name);

    printf("\nWriting Fits File!");

    //runs the writing program that creates the output .fit
    //file for the user.
    error = write_output_fits (p_fp_out_fits, p_output_array,
p_out_num_rows);

    //This is the point where it usually goes wrong
    printf("\nClosing Fits File!");
    fits_close_file (*p_fp_out_fits, &status);
    if (status!=0)
        {
        fits_report_error (stdout, status);
        return (status);
        }

    printf("\nFinished!");
    return (error);


}

//standardised final writing stage
int make_out_fits ( fitsfile **p_fp_out_fits, char *p_file_name)
{
    //error handling
    int error = 0;
    int status = 0;
    int *p_status = &status;

    //creates name and format files for the new table
    char ra_name[16] = "ra";
    char *p_ra_name = & ra_name[0];
    char form[3] = "1D";
    char *p_form = &form[0];

    //creates a file
    fits_create_file (p_fp_out_fits, p_file_name, p_status);
    if (status!=0)
        {
        fits_report_error (stdout, status);
        return (status);
        }

    //creates a fits data table
    fits_create_tbl (*p_fp_out_fits, BINARY_TBL, 0, 1, &p_ra_name, &p_form,
NULL, NULL, p_status);
    if (status!=0)
        {
        fits_report_error (stdout, status);
        return (status);
        }
    printf ("\nNew FITS table created!\n");

    return (error);
}

int write_output_fits(    fitsfile **p_fp_out_fits,
                        double **p_output_array,
                        long int *p_out_num_rows)

{
    //error handling
    int error = 0;
    int status = 0;
    int *p_status = &status;

    //declares an index for control of writing and
    //column numbers the array is to write to
    long int output_index =0;
    long int row_index = output_index +1;
    int new_ra_col = 1;

    //goes through the array line by line writing it to the file.
    for(output_index = 0 ; output_index < *p_out_num_rows ; output_index ++)
    {
        row_index = output_index +1;

        //writes to the file
        fits_write_col (*p_fp_out_fits, TDOUBLE, new_ra_col, row_index, 1,
1, (((*p_output_array)+output_index)/*->ra*/), p_status);

        if (status!=0)
            {
            fits_report_error (stdout, status);
            return (status);
            }

        printf ("%lf\n", *((*p_output_array)+output_index));
    }

    return (error);
}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listmgr.nrao.edu/pipermail/fitsbits/attachments/20111018/85f0d40c/attachment.html>