CCD proc crashes on 2.12.2-EXPORT on 64 bit opteron
Greg Hennessy wrote on Jul 14, 2006
I have a strange problem in that ccdproc on two similar computers act in different fashions. Both computers are running IRAF 2.12.2 on Fedora Core 5. Both machines are AMD Opteron dual processors. I'm sure I'm running the exact same software on both since I did an rsync of the binaries.
On the computer that isn't behaving properly, CCDPROC processes
about five files then stops with
ERROR: floating point invalid operation
A temp file is created that seems to be corrupt.
Warning: FXF: EOF encountered while reading FITS file (tmp4422i.fits)
The exact same software works fine on a different computer. Does anyone have any sugguestions on how to find/fix the problem? I've not had any thing similar happen before.
On the computer that isn't behaving properly, CCDPROC processes
about five files then stops with
ERROR: floating point invalid operation
A temp file is created that seems to be corrupt.
Warning: FXF: EOF encountered while reading FITS file (tmp4422i.fits)
The exact same software works fine on a different computer. Does anyone have any sugguestions on how to find/fix the problem? I've not had any thing similar happen before.
Mike Fitzpatrick wrote on Jul 14, 2006
(Very rarely) Something like this could be explained by a hardware problem, e.g. a bad memory SIMM, flaky disk, /tmp is full, etc. If the same data can be processed on another machine it likely isn't data-related but take note if the data is local to one machine and NFS mounted on the other. The 'dmesg' command will print the system logfile where hardware/kernel messages are stored, and sometimes simply rebooting the machine to show it who's boss works (pagan costume not always required but helpful).
Otherwise, does the order the images are processed matter? Is it always the fifth image regardless of the list, or only that particular fifth image? The 32-bit binaries should work fine on Opterons but to be sure you're not running into a pointer problem, verify that the 'stacksize' limit isn't set, e.g.
Either command should return 'unlimited', if not then reset it in your .cshrc/.bashrc file with "limit stacksize unlimited" (csh) or "ulimit -s unlimited" (bash). The CL startup script should already be doing this, but check the hlb$cl.csh to be sure (you need this in a .cshrc and not .login because the cl.csh reads this at startup).
Other than that I'm stumped. Hopefully Frank or another Opteron user will see something familiar. Hope thi shelps.
Cheers,
-Mike
Otherwise, does the order the images are processed matter? Is it always the fifth image regardless of the list, or only that particular fifth image? The 32-bit binaries should work fine on Opterons but to be sure you're not running into a pointer problem, verify that the 'stacksize' limit isn't set, e.g.
csh_prompt% limit stacksize
bash_prompt% ulimit -s
Either command should return 'unlimited', if not then reset it in your .cshrc/.bashrc file with "limit stacksize unlimited" (csh) or "ulimit -s unlimited" (bash). The CL startup script should already be doing this, but check the hlb$cl.csh to be sure (you need this in a .cshrc and not .login because the cl.csh reads this at startup).
Other than that I'm stumped. Hopefully Frank or another Opteron user will see something familiar. Hope thi shelps.
Cheers,
-Mike
Greg Hennessy wrote on Jul 14, 2006
The data have been "rsync"ed between the two machines, so no NFS involved.
The number of frames it does before exiting with the error isn't constantly 5, today it seems to be doing about 8-10 before exiting. The exit message is always the same sort:
ERROR: floating point invalid operation
dmesg doesn't so anythign obvious,
mtrr: type mismatch for f0000000,4000000 old: write-back new: write-combining
I'll have to google to see what that message means.
limit stacksize unlimited was done (I'm a tcsh weenie), but no change.
The order doesn't seem to matter. And executing a new instance of "ccdproc" will happily process the image it bombed out on before, and then it moves forward. I can eventually get my data reduced, but its way more of a pain in the tucas than it *should* be. Insert obligatory "Welcome to IRAF" joke.
I'm about at the sacrifice a chicken stage.
The number of frames it does before exiting with the error isn't constantly 5, today it seems to be doing about 8-10 before exiting. The exit message is always the same sort:
ERROR: floating point invalid operation
dmesg doesn't so anythign obvious,
mtrr: type mismatch for f0000000,4000000 old: write-back new: write-combining
I'll have to google to see what that message means.
limit stacksize unlimited was done (I'm a tcsh weenie), but no change.
The order doesn't seem to matter. And executing a new instance of "ccdproc" will happily process the image it bombed out on before, and then it moves forward. I can eventually get my data reduced, but its way more of a pain in the tucas than it *should* be. Insert obligatory "Welcome to IRAF" joke.
I'm about at the sacrifice a chicken stage.
Last post on Jul 14, 2006