[daip] Two on-line FILLMs on same machine -> trouble?
Michael Rupen
mrupen at aoc.nrao.edu
Wed Jul 16 09:37:05 EDT 2003
Good morning DAIP,
I was running two on-line FILLMs on the same machine, which I suspect is
verboten. In any case it didn't work, and I thought I'd report this, in case
it's supposed to :} FILLM #1 was begun yesterday (15jul03) morning on verdi
[Linux box], at around 8am; FILLM #2, at around 9:30am. FILLM #1 died saying it
didn't find any data, with an FLMCLS error on flushing output. FILLM #2
worked perfectly, ending several hours after FILLM #1's demise. According to
the VLA logs both projects were observed without any difficulties. Off-line
FILLM, with the same inputs as FILLM #1 but version TST rather than ONLINE,
worked spiffily.
I append (1) inputs and output for FILLM #1, (2) inputs and output for
FILLM #2 (search for #2 to jump there), and (3) inputs and output for off-line
FILLM which correctly loaded data corresponding to FILLM #1 (search for
OFF-LINE to jump there).
Cheers,
Michael
--------
FILLM #1
--------
* Started first
* Failed when it started actually loading data (about 9 hours later!),
with
FILLM No data found meeting selection criteria!
FILLM FLMCLS: ERROR 1 FLUSHING OUTPUT
FILLM FLMCLS: ERROR OCCURED ON STREAM 1
FILLM Purports to die of UNNATURAL causes
FILLM verdi 31DEC03 TST: Cpu= 0.0 Real= 36329
* The other FILLM sailed through this period and ran successfully until
it finished this morning (16jul03) at 04:45am, many (11?) hours after
FILLM #1 failed.
* inputs to FILLM #1:
AIPS 2: Found in Version=ONLINE
AIPS 2: FILLM Task to read a VLA archive tape
AIPS 2: Adverbs Values Comments
AIPS 2: ----------------------------------------------------------------
AIPS 2: INTAPE 1 Tape drive number
AIPS 2: INFILE ' '
AIPS 2: Non-blank: disk file name
AIPS 2: NFILES 0 Number of files to advance
AIPS 2: BAND ' ' Freq. band (4,P,L,C,X,U,K,Q)
AIPS 2: QUAL -1 Source qualifier -1=>all
AIPS 2: CALCODE ' ' Calibrator code
AIPS 2: VLAOBS 'AK569 ' VLA obs. pgm. name ('AZ99')
AIPS 2: VLAMODE ' ' VLA obs. mode (' '=any)
AIPS 2: REFDATE ' ' Reference date. 'yyyymmdd'
AIPS 2: TIMERANG *all 0 Timerange selected
AIPS 2: BCHAN 1 First spectral channel
AIPS 2: ECHAN 0 Highest spectral channel
AIPS 2: OUTNAME ' ' Output UV file name (name)
AIPS 2: OUTSEQ 0 Output UV file name (seq. #)
AIPS 2: OUTDISK 1 Output UV file disk unit #.
AIPS 2: DOUVCOMP -1 1 (T) => compressed data
AIPS 2: DOALL 1 1 (T) => write all data.
AIPS 2: DOCONCAT -1 1 (T) => append data to old
AIPS 2: files
AIPS 2: NCOUNT 0 Number of files to read
AIPS 2: DOWEIGHT 1 Use nominal sensitivity to
AIPS 2: scale weights?
AIPS 2: DOACOR 0 Load autocorrelation data?
AIPS 2: CPARM 0 0 User options.
AIPS 2: 0 0 1 => Avg. time (seconds)
AIPS 2: 0 0 2 => bitmap (see help)
AIPS 2: 0 0.1667 = 16 for planets & Sun
AIPS 2: 0 0 3 => Max. OK IF status
AIPS 2: <=3 => 3
AIPS 2: 4 => Control dropping of
AIPS 2: shadowed data
AIPS 2: < 0 => no shadow check
AIPS 2: 0 => 25 m limit
AIPS 2: > 0 => Shadow limit is
AIPS 2: CPARM(4) in meters
AIPS 2: 5 = channel code (see HELP)
AIPS 2: 6 => Subarray number-see HELP
AIPS 2: 7 => FQ entry tolerance (kHz)
AIPS 2: (see HELP)
AIPS 2: 8 => CL table time incr. min
AIPS 2: 9 => TY table time incr. min
AIPS 2: 10=> Calibrator avg. time
AIPS 2: DPARM *all 0 Selection by Frequency
AIPS 2: 1+2 => A Frequency (Hz)
AIPS 2: 3+4 => B Frequency (Hz)
AIPS 2: 5+6 => C Frequency (Hz)
AIPS 2: 7+8 => D Frequency (Hz)
AIPS 2: 9 => tolerance for A&C
AIPS 2: 10 => tolerance for B&D
AIPS 2: BPARM *all 0 Opacity and Gain curve
AIPS 2: control (see help)
AIPS 2: IN2FILE ' '
AIPS 2: Antenna gains file
* PRTMSG for FILLM #1:
verdi AIPS (31DEC03) 11 16-JUL-2003 07:06:37 Page 1
Pops Prior Date Time Task Messages for user 11
2 5 15-JUL-2003 08:02:24 FILLM Task FILLM (release of 31DEC03) begins
2 4 15-JUL-2003 08:02:24 FILLM Shadow flag limit = 2.500E+01 meters.
2 3 15-JUL-2003 08:02:24 FILLM Opacity correction in CL table weighted average of weather and
2 3 15-JUL-2003 08:02:24 FILLM season. Weight for weather = 0.50
2 3 15-JUL-2003 08:02:24 FILLM Gain curve correction in CL table read from file,
2 3 15-JUL-2003 08:02:24 FILLM with variation as function of antenna and band.
2 3 15-JUL-2003 08:02:24 FILLM Welcome to real-time FILLM!
2 8 15-JUL-2003 08:02:35 FILLM ZTPOP2: olopen -1
2 5 15-JUL-2003 08:02:37 FILLM tape file # 1, start date/time = 20030715/14:02:50
2 8 15-JUL-2003 08:35:31 FILLM ZTPMI2: End-of-data: try to reopen
2 8 15-JUL-2003 08:37:22 FILLM ZTPMI2: olopen -1
2 8 15-JUL-2003 08:48:11 FILLM ZTPMI2: End-of-data: try to reopen
2 8 15-JUL-2003 08:50:02 FILLM ZTPMI2: olopen -1
2 8 15-JUL-2003 09:00:51 FILLM ZTPMI2: End-of-data: try to reopen
2 8 15-JUL-2003 09:02:42 FILLM ZTPMI2: olopen -1
.....
2 8 15-JUL-2003 12:05:10 FILLM ZTPMI2: End-of-data: try to reopen
2 8 15-JUL-2003 12:06:50 FILLM ZTPMI2: olopen -2
2 2 15-JUL-2003 18:07:50 FILLM
2 2 15-JUL-2003 18:07:50 FILLM *** ATTENTION - default: loading subarray 1 ***
2 2 15-JUL-2003 18:07:50 FILLM
2 5 15-JUL-2003 18:07:50 FILLM MCINI: Processing Correlator Code ' ' with 27 antennas.
2 3 15-JUL-2003 18:07:50 FILLM MCINI - Continuum, 2 compatible IFs
2 8 15-JUL-2003 18:07:53 FILLM No data found meeting selection criteria!
2 8 15-JUL-2003 18:07:53 FILLM FLMCLS: ERROR 1 FLUSHING OUTPUT
2 8 15-JUL-2003 18:07:53 FILLM FLMCLS: ERROR OCCURED ON STREAM 1
2 3 15-JUL-2003 18:07:53 FILLM Purports to die of UNNATURAL causes
2 5 15-JUL-2003 18:07:53 FILLM verdi 31DEC03 TST: Cpu= 0.0 Real= 36329
* Screen capture for FILLM #1:
Connecting...Connected.
bad recv status of: No such file or directory
The aoc_clnt process has terminated unexpectedly.error on calling setitimer: Interrupted system call
FILLM2: ZTPMI2: olopen -1
FILLM2: ZTPMI2: End-of-data: try to reopen
Connecting...Connected.
bad recv status of: No such file or directory
The aoc_clnt process has terminated unexpectedly.error on calling setitimer: Interrupted system call
FILLM2: ZTPMI2: olopen -1
FILLM2: ZTPMI2: End-of-data: try to reopen
FILLM2: ZTPMI2: olopen -2
7 7 8 8 9 9 10 10 11 11 12 12 13 14 14 15 15 16 16 17 17 18 18 19 19 20 20 21 21 22 22 23 23 24 24 25 25 26 26 27 27 28 28 29 29 30 30 31 31 32 32 33 33 34 34 35 35 36 36 37 37 38 38 39 39 40 40 41 41 42 42 43 43 44 44 45 45 46 46 47 47 48 48 49 49 50 50 51 51 52 52 53 53 54 54 55 55 56 56 57 57 58 58 59 59 60 60 61 61 63 62 64 63 65 64 66 65 67 66 68 67 69 68 70 69 71 70 72 71 74 72 75 73 76 74 77 75 78 76 79 77 80 78 82 79 84 80 85 81 86 82 87 83 88 84 89 85 90 86 91 87 92 88 93 89 94 90 95 91 96 92 97 93 98 94 99 95 100 96 101 97 102 98 103 99 104 100 105 101 106 102 107 103 108 104 109 105
....
138 2143 2139 2144 2140 2145 2141 2146 2142 2147 2143 2148 2144 2149 2145 2150 2146 2151 2147 2152 2148 2153 2149 2154 2150 2155 2151 2156 2152 2157 2153 2158 2154 2160 2161 2162 2162 2163 2163 2164 2164 2165 2165 2166 2166 2167 2167 2168 2168 2170 2169 2171 2170 2172 2171 2173 2172 2174 2173 2175 2174 2176 2175 2177 2176 FILLM2:
FILLM2: *** ATTENTION - default: loading subarray 1 ***
FILLM2:
FILLM2: MCINI: Processing Correlator Code ' ' with 27 antennas.
FILLM2: MCINI - Continuum, 2 compatible IFs
2178 FILLM2: No data found meeting selection criteria!
FILLM2: FLMCLS: ERROR 1 FLUSHING OUTPUT
FILLM2: FLMCLS: ERROR OCCURED ON STREAM 1
FILLM2: Purports to die of UNNATURAL causes
FILLM2: verdi 31DEC03 TST: Cpu= 0.0 Real= 36329
--------
FILLM #2
--------
* Started second
* Worked perfectly, ending many (11?) hours after FILLM #1 failed.
* Inputs to FILLM #2: identical to FILLM #1 except for INTAPE and VLAOBS
AIPS 1: Found in Version=ONLINE
AIPS 1: FILLM Task to read a VLA archive tape
AIPS 1: Adverbs Values Comments
AIPS 1: ----------------------------------------------------------------
AIPS 1: INTAPE 2 Tape drive number
AIPS 1: INFILE ' '
AIPS 1: Non-blank: disk file name
AIPS 1: NFILES 0 Number of files to advance
AIPS 1: BAND ' ' Freq. band (4,P,L,C,X,U,K,Q)
AIPS 1: QUAL -1 Source qualifier -1=>all
AIPS 1: CALCODE ' ' Calibrator code
AIPS 1: VLAOBS 'AR523 ' VLA obs. pgm. name ('AZ99')
AIPS 1: VLAMODE ' ' VLA obs. mode (' '=any)
AIPS 1: REFDATE ' ' Reference date. 'yyyymmdd'
AIPS 1: TIMERANG *all 0 Timerange selected
AIPS 1: BCHAN 1 First spectral channel
AIPS 1: ECHAN 0 Highest spectral channel
AIPS 1: OUTNAME ' ' Output UV file name (name)
AIPS 1: OUTSEQ 0 Output UV file name (seq. #)
AIPS 1: OUTDISK 1 Output UV file disk unit #.
AIPS 1: DOUVCOMP -1 1 (T) => compressed data
AIPS 1: DOALL 1 1 (T) => write all data.
AIPS 1: DOCONCAT -1 1 (T) => append data to old
AIPS 1: files
AIPS 1: NCOUNT 0 Number of files to read
AIPS 1: DOWEIGHT 1 Use nominal sensitivity to
AIPS 1: scale weights?
AIPS 1: DOACOR 0 Load autocorrelation data?
AIPS 1: CPARM 0 0 User options.
AIPS 1: 0 0 1 => Avg. time (seconds)
AIPS 1: 0 0 2 => bitmap (see help)
AIPS 1: 0 0.1667 = 16 for planets & Sun
AIPS 1: 0 0 3 => Max. OK IF status
AIPS 1: <=3 => 3
AIPS 1: 4 => Control dropping of
AIPS 1: shadowed data
AIPS 1: < 0 => no shadow check
AIPS 1: 0 => 25 m limit
AIPS 1: > 0 => Shadow limit is
AIPS 1: CPARM(4) in meters
AIPS 1: 5 = channel code (see HELP)
AIPS 1: 6 => Subarray number-see HELP
AIPS 1: 7 => FQ entry tolerance (kHz)
AIPS 1: (see HELP)
AIPS 1: 8 => CL table time incr. min
AIPS 1: 9 => TY table time incr. min
AIPS 1: 10=> Calibrator avg. time
AIPS 1: DPARM *all 0 Selection by Frequency
AIPS 1: 1+2 => A Frequency (Hz)
AIPS 1: 3+4 => B Frequency (Hz)
AIPS 1: 5+6 => C Frequency (Hz)
AIPS 1: 7+8 => D Frequency (Hz)
AIPS 1: 9 => tolerance for A&C
AIPS 1: 10 => tolerance for B&D
AIPS 1: BPARM *all 0 Opacity and Gain curve
AIPS 1: control (see help)
AIPS 1: IN2FILE ' '
AIPS 1: Antenna gains file
* PRTMSG for FILLM #2:
Note that the other FILLM ran successfully until it finished this morning
(16jul03) at 4:45am:
1 8 15-JUL-2003 12:02:36 FILLM ZTPMI2: End-of-data: try to reopen
1 8 15-JUL-2003 12:04:18 FILLM ZTPMI2: olopen -1
1 8 15-JUL-2003 12:04:18 FILLM ZTPMI2: End-of-data: try to reopen
1 8 15-JUL-2003 12:06:09 FILLM ZTPMI2: olopen -1
1 5 15-JUL-2003 12:06:29 FILLM tape file # 1, start date/time = 20030715/18:06:20
1 2 16-JUL-2003 01:40:53 FILLM
1 2 16-JUL-2003 01:40:53 FILLM *** ATTENTION - default: loading subarray 1 ***
1 2 16-JUL-2003 01:40:53 FILLM
1 5 16-JUL-2003 01:40:53 FILLM MCINI: Processing Correlator Code ' ' with 27 antennas.
1 3 16-JUL-2003 01:40:53 FILLM MCINI - Continuum, 2 compatible IFs
1 3 16-JUL-2003 01:40:56 FILLM Program = AR523 ; Tape revision number = 26.
1 2 16-JUL-2003 01:40:56 FILLM Create 20030716 .X BAND. 1 (UV) on disk 1 cno 481
1 4 16-JUL-2003 01:40:56 FILLM Ref. date = 20030716 A-C = 8.435100 B-D = 8.485100 GHz
1 6 16-JUL-2003 01:40:56 FILLM FLMFQ: FQ entry tolerance = 1.000E+02 1.000E+02
...etc., all working as usual; and ending with
1 4 16-JUL-2003 04:34:06 FILLM Appending new data to: 20030716 .L BAND. 1 disk 1
1 4 16-JUL-2003 04:34:06 FILLM Found 0319+415 : 0 50.000 MHz at IAT 0/ 10:34:20.0
1 3 16-JUL-2003 04:34:16 FILLM Zenith opacity set to 0.010
1 5 16-JUL-2003 04:43:59 FILLM Read 834848 visibilities from999 files
1 3 16-JUL-2003 04:43:59 FILLM Appears to have ended successfully
--------------
OFF-LINE FILLM
--------------
* Run with same inputs as FILLM #1, but version TST and from exabyte
DN18568
* Inputs:
AIPS 2: FILLM Task to read a VLA archive tape
AIPS 2: Adverbs Values Comments
AIPS 2: ----------------------------------------------------------------
AIPS 2: INTAPE 3 Tape drive number
AIPS 2: INFILE ' '
AIPS 2: Non-blank: disk file name
AIPS 2: NFILES 0 Number of files to advance
AIPS 2: BAND ' ' Freq. band (4,P,L,C,X,U,K,Q)
AIPS 2: QUAL -1 Source qualifier -1=>all
AIPS 2: CALCODE ' ' Calibrator code
AIPS 2: VLAOBS 'AK569 ' VLA obs. pgm. name ('AZ99')
AIPS 2: VLAMODE ' ' VLA obs. mode (' '=any)
AIPS 2: REFDATE ' ' Reference date. 'yyyymmdd'
AIPS 2: TIMERANG *all 0 Timerange selected
AIPS 2: BCHAN 1 First spectral channel
AIPS 2: ECHAN 0 Highest spectral channel
AIPS 2: OUTNAME ' ' Output UV file name (name)
AIPS 2: OUTSEQ 0 Output UV file name (seq. #)
AIPS 2: OUTDISK 1 Output UV file disk unit #.
AIPS 2: DOUVCOMP -1 1 (T) => compressed data
AIPS 2: DOALL 1 1 (T) => write all data.
AIPS 2: DOCONCAT -1 1 (T) => append data to old
AIPS 2: files
AIPS 2: NCOUNT 0 Number of files to read
AIPS 2: DOWEIGHT 1 Use nominal sensitivity to
AIPS 2: scale weights?
AIPS 2: = 10, use Memo 108 weights
AIPS 2: DOACOR 0 Load autocorrelation data?
AIPS 2: CPARM 0 0 User options.
AIPS 2: 0 0 1 => Avg. time (seconds)
AIPS 2: 0 0 2 => bitmap (see help)
AIPS 2: 0 0.1667 = 16 for planets & Sun
AIPS 2: 0 0 3 => Max. OK IF status
AIPS 2: <=3 => 3
AIPS 2: 4 => Control dropping of
AIPS 2: shadowed data
AIPS 2: < 0 => no shadow check
AIPS 2: 0 => 25 m limit
AIPS 2: > 0 => Shadow limit is
AIPS 2: CPARM(4) in meters
AIPS 2: 5 = channel code (see HELP)
AIPS 2: 6 => Subarray number-see HELP
AIPS 2: 7 => FQ entry tolerance (kHz)
AIPS 2: (see HELP)
AIPS 2: 8 => CL table time incr. min
AIPS 2: 9 => TY table time incr. min
AIPS 2: 10=> Calibrator avg. time
AIPS 2: DPARM *all 0 Selection by Frequency
AIPS 2: 1+2 => A Frequency (Hz)
AIPS 2: 3+4 => B Frequency (Hz)
AIPS 2: 5+6 => C Frequency (Hz)
AIPS 2: 7+8 => D Frequency (Hz)
AIPS 2: 9 => tolerance for A&C
AIPS 2: 10 => tolerance for B&D
AIPS 2: BPARM *all 0 Opacity and Gain curve
AIPS 2: control (see help)
AIPS 2: IN2FILE ' '
AIPS 2: Antenna gains file
* Output:
FILLM2: Task FILLM (release of 31DEC03) begins
FILLM2: Shadow flag limit = 2.500E+01 meters.
FILLM2: Opacity correction in CL table weighted average of weather and
FILLM2: season. Weight for weather = 0.50
FILLM2: Gain curve correction in CL table read from file,
FILLM2: with variation as function of antenna and band.
FILLM2: tape file # 2, start date/time = 20030715/12:45:25
FILLM2:
FILLM2: *** ATTENTION - default: loading subarray 1 ***
FILLM2:
FILLM2: MCINI: Processing Correlator Code ' ' with 27 antennas.
FILLM2: MCINI - Continuum, 2 compatible IFs
FILLM2: Program = AK569 ; Tape revision number = 26.
FILLM2: Create 20030716 .C BAND. 2 (UV) on disk 1 cno 487
FILLM2: Ref. date = 20030716 A-C = 4.885100 B-D = 4.835100 GHz
FILLM2: FLMFQ: FQ entry tolerance = 1.000E+02 1.000E+02
FILLM2: Found 1331+305 : 0 50.000 MHz at IAT 0/ 00:08:10.0
FILLM2: Zenith opacity set to 0.010
FILLM2: Trying to access VLA antenna gains file...
FILLM2: ZTXOP2: using translated file name =
FILLM2: ZTXOP2: /home/AIPS/TEXT/IONS/VLA.GAINS
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:13:00.0
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:14:43.3
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:15:43.3
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:17:13.3
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:18:13.3
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:19:43.3
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:20:43.3
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:22:13.3
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:23:13.3
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:24:43.3
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:25:43.3
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:27:13.3
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:28:13.3
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:29:43.3
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:30:43.3
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:32:10.0
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:33:10.0
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:34:40.0
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:35:40.0
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:37:10.0
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:38:10.0
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:39:40.0
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:40:40.0
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:42:10.0
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:43:10.0
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:44:43.3
FILLM2: Found 14033-41230 : 0 50.000 MHz at IAT 0/ 00:45:43.3
FILLM2: Found 13529-44127 : 0 50.000 MHz at IAT 0/ 00:47:13.3
FILLM2: Read 199048 visibilities from 5 files
FILLM2: Appears to have ended successfully
FILLM2: verdi 31DEC03 TST: Cpu= 3.1 Real= 347
================================================================================
More information about the Daip
mailing list