View on GitHub

IRAF Community Distribution

IRAF maintained by the community

Home | Installation | Packages | X11IRAF | PyRAF | Forum

ccdproc PANIC/segmentation violation

vmagalhaes wrote on May 01, 2012

I have a instalation of 2.16 with both 32 and 64 bits mac os x binaries.
i have been trying to correct some flat images with an averaged bias image.
When i try i get the following error messages:
64 bits:
ecl> ccdproc
List of CCD images to correct (flat*):
ERROR: segmentation violation
32 bits:
ecl> ccdproc
List of CCD images to correct (flat*):

PANIC in `/iraf/noao/bin.macosx/x_ccdred.e': Memory fwa has been corrupted

In neither cases does iraf crash or close, its working fine.
My belief is that this is a problem with x_ccdred.e
is there a patch for this executable?
or a recipe for me to recompile it locally?

Thank you for any advice.

Mike Fitzpatrick wrote on May 01, 2012

There's been a similar report recently (http://iraf.net/phpBB2/viewtopic.php?t=90895) but no followup yet that identified the problem or the fix. Would it be possible for you to upload some sample images and parameters to the anonftp at ftp://iraf.noao.edu/pub so we can try to reproduce the problem?

You can be sure you're running the latest version by doing:


    % cd $iraf
    % make latest


The default CL in v2.16 should give you a "vocl>" prompt not "ecl>", did you deliberately start the old CL or is there some chance this is a v2.15 system (e.g. via some old scisoft installation)? You can do "show version" to find out.

vmagalhaes wrote on May 01, 2012

Thanks for the rapid response.

My script for starting iraf deliberately started ecl instead of cl
I changed it and now it starts in vocl.
But this change didn't make any difference...

i tried updating with the make latest command.
it downloaded the new versions of some parts of IRAF but still no difference...

I tried redownloading all the distribution and still got the same error...
I also tried using other data, but still from the same observational run, and got the same error...

i'm trying to upload a file with the images but the pub ftp tells me i don't have permission...
do you have another alternative?

Mike Fitzpatrick wrote on May 01, 2012

I asked about the ecl vs. vocl thing only to be sure we're talking about v2.16, the error is in the CCDPROC binary itself and so this shouldn't matter.

The anonftp doesn't allow you to create subdirs you can write to, just upload everything to the toplevel ftp://iraf.noao.edu/pub directory or else send me a URL where I can nab it (or email fitz@iraf.net).

vmagalhaes wrote on May 01, 2012

The problem with uploading to the ftp was fixed when i updated filezilla...
the file is there with the name ccdproc_error.tar.gz

it contains an averaged bias image created with zerocombine.
and one of the flat images i was trying to use.

Mike Fitzpatrick wrote on May 01, 2012

I can't seem to reproduce the problem. Would you mind sending the parameters you used (e.g. the output of "dpar ccdproc") as well as the command

cl> dir noaobin$x_ccdred.e long+


so I can see what version you're using?

Francisco Valdes wrote on May 01, 2012

I made an attempt to reproduce the problem but no luck. This was on an iMac.


ccdred> =cl.version
IRAF V2.16 March 2012
ccdred> ccdproc flat_001.fits output=test.fits zero=bias_ave.fits
ccdred> cat logfile
flat_001.fits: May 10 15:35 Zero level correction image is bias_ave.fits
ccdred> imstat *.fits
#               IMAGE      NPIX      MEAN    STDDEV       MIN       MAX
        bias_ave.fits   4194304     886.2    0.4079     884.5     905.6
        flat_001.fits   4194304    20059.     983.4     1086.    30714.
            test.fits   4194304    19173.     983.5     199.7    29827.
ccdred> lpar ccdred ccdproc
    ccdred:
   (pixeltype = "real real")    Output and calculation pixel datatypes
     (verbose = no)             Print log information to the standard output?
     (logfile = "logfile")      Text log file
    (plotfile = "")             Log metacode plot file
      (backup = "")             Backup directory or prefix
  (instrument = "")             CCD instrument file
      (ssfile = "subsets")      Subset translation file
    (graphics = "stdgraph")     Interactive graphics output device
      (cursor = "")             Graphics cursor input
     (version = "2: October 1987") 
        (mode = "ql")           
      ($nargs = 0)              
    ccdproc:
       images = "flat_001.fits" List of CCD images to correct
      (output = "")             List of output CCD images
     (ccdtype = "")             CCD image type to correct
   (max_cache = 0)              Maximum image caching memory (in Mbytes)
      (noproc = no)             List processing steps only?\n
      (fixpix = no)             Fix bad CCD lines and columns?
    (overscan = no)             Apply overscan strip correction?
        (trim = no)             Trim the image?
     (zerocor = yes)            Apply zero level correction?
     (darkcor = no)             Apply dark count correction?
     (flatcor = no)             Apply flat field correction?
    (illumcor = no)             Apply illumination correction?
   (fringecor = no)             Apply fringe correction?
     (readcor = no)             Convert zero level image to readout correction?
     (scancor = no)             Convert flat field image to scan correction?\n
    (readaxis = "line")         Read out axis (column|line)
     (fixfile = "")             File describing the bad lines and columns
     (biassec = "")             Overscan strip image section
     (trimsec = "")             Trim data section
        (zero = "")             Zero level calibration image
        (dark = "")             Dark count calibration image
        (flat = "")             Flat field images
       (illum = "")             Illumination correction images
      (fringe = "")             Fringe correction images
  (minreplace = 1.)             Minimum flat field value
    (scantype = "shortscan")    Scan type (shortscan|longscan)
       (nscan = 1)              Number of short scan lines\n
 (interactive = no)             Fit overscan interactively?
    (function = "legendre")     Fitting function
       (order = 1)              Number of polynomial terms or spline pieces
      (sample = "*")            Sample points to fit
    (naverage = 1)              Number of sample points to combine
    (niterate = 1)              Number of rejection iterations
  (low_reject = 3.)             Low sigma rejection factor
 (high_reject = 3.)             High sigma rejection factor
        (grow = 0.)             Rejection growing radius
        (mode = "ql")           

marco.03 wrote on May 01, 2012

hi
I am having a similar problem with ubuntu 64bit 12.04 (kernel 3.2.0-38 ).
I have installed the linux64 binaries.
Running ccdproc just to trim any image gives me the segmentation violation error.
I have tried running d_trace before ccdproc. This is what I get

        ----- task cl - line 42 ( ) -----
      42      40+4:     call            ccdproc
      42      44+7:     pushconst       `dati1.fits'
      42      51+4:     indirposset     0
      42      55+8:     pushconst       `[5:2000,200:500]'
      42      63+4:     indirabsset     trimsec
      42      67+3:     exec
        ----- exec external task ccdproc - line 42 -----
        ----- connect to noaobin$x_ccdred.e -----
            ccdproc\nimages="dati1.fits"\noutput=""\nccdtype=" "\n
            max_cache=0\nnoproc=no\nfixpix=no\noverscan=no\ntrim=yes\n
            zerocor=no\ndarkcor=no\nflatcor=no\nillumcor=no\n
            fringecor=no\nreadcor=no\nscancor=no\nreadaxis="line"\n
            fixfile=""\nbiassec=""\ntrimsec="[5:2000,200:500]"\nzero=""\n
            dark=""\nflat=""\nillum=""\nfringe=""\nminreplace=1.\n
            scantype="shortscan"\nnscan=1\ninteractive=no\n
            function="legendre"\norder=1\nsample="*"\nnaverage=1\n
            niterate=1\nlow_reject=3.\nhigh_reject=3.\ngrow=0.\n
            mode="ql"\n$nargs=1\n
        --------------------------------
        ----- task ccdproc - line 32 ( ) -----
      32      73+5:     inspect         instrument
      32      78+3:     end
        ----- task ccdproc - line 33 ( ) -----
      33      73+5:     inspect         pixeltype
      33      78+3:     end
        ----- task ccdproc - line 34 ( ) -----
      34      73+5:     inspect         pixeltype
      34      78+3:     end
        ----- task ccdproc - line 35 ( ) -----
      35      73+4:     inspect         verbose
      35      77+3:     end
        ----- task ccdproc - line 36 ( ) -----
      36      73+4:     inspect         logfile
      36      77+3:     end
        ----- task ccdproc - line 37 ( ) -----
      37      73+4:     inspect         backup
      37      77+3:     end
        ----- task ccdproc - line 38 ( ) -----
      38      73+4:     call            error
      38      77+5:     pushconst       501
      38      82+4:     posargset       0
      38      86+8:     pushconst       `segmentation violation'
      38      94+4:     posargset       1
      38      98+3:     exec
        ----- exec builtin error : line 38 -----
        ----- disconnect noaobin$x_ccdred.e -----
ERROR: segmentation violation
cl_error: abort=1  beep=1  trace=1  flpr=1
cl_error: code=501 do_err=1 errtype=1/1 task='cl'

In this particular case the image is 2048x1034. but this happens with every image I try to crop.
I have already tried 'make latest', but I get the same error.

Marco

Mike Fitzpatrick wrote on May 01, 2012

As you see from the previous posts, we weren't able to reproduce the problem locally on mac systems, perhaps linux64 will be different. Could you upload a sample image to the anonftp at ftp://iraf.noao.edu/pub in case the problem is data related and we'll try again.

marco.03 wrote on May 01, 2012

I have uploaded the ccdproc_sv.tar.gz. Howevere I think the problem is not data related, because I get the same error every time I try to crop any image.

Vallery Stanishev wrote on May 01, 2012

a couple of week ago I installed IRAF2.16 on Fedora 18 64-bit. For fat I was doing photometry on already pre-redduced images and everything was working fine. Today I tried oevrscan and trim corrections with ccdproc and I get exactly the same problem.


vocl> ccdpro KXSA0093 
KXSA0093: Mar  5 17:19 Trim data section is [53:2100,1:2048]
Fit overscan vector for KXSA0093 interactively (yes): 
KXSA0093: Mar  5 17:19 Overscan section is [53:2100,2057:2100] with mean=1162.382
ERROR: segmentation violation


However:

vocl> ccdpro KXSA0093 out=test
KXSA0093: Mar  5 17:20 Trim data section is [53:2100,1:2048]
Fit overscan vector for KXSA0093 interactively (yes): 
KXSA0093: Mar  5 17:20 Overscan section is [53:2100,2057:2100] with mean=1162.382


works fine, but when I do:
vocl> imdel test
ERROR: segmentation violation


it does not work again.

So, maybe the problem is that for some reason 'imdelete' is not working and when ccdrpoc tries to override the images it fails. I tried to imdelete some of the older, prereduced images on which I was doing photometry and imdelete failed.

Vallery Stanishev wrote on May 01, 2012

Some more issues popped up the more I use IRAF 2.16 on Fedora 18. Aperture photometry seems to work fine, but when I tried PSF, it fails to write the computed PSF. Then I used 'wcsctran' but it claims that the input coordinate files do not exist, but they surely exist. Then I moved all the jobs to another laptop with Fedora 17 instead of Fedora 18, and it all worked fine.

I do not understand why is this happening. Sure, the new laptop has SSD, ext4, while the old normal HDD, ext3, but this should not matter.

Mike Fitzpatrick wrote on May 01, 2012

In reply to marco.03:

Thanks for the data, however I am still not able to reproduce the problem using either the v2.15, v2.16 or current development versions. I do notice that you have the 'ccdtype' parameter set to a space and not a null string (i.e. " " and not ""), but this doesn't seem to make a difference. Is this an IRAF system you got from scisoft or some other source? Was it recompiled locally in any way? Are you likewise on a Fedora 18 or similar system as vall (the kernel version is very recent)?? Could you both please post the output of the command

cl> dir noaobin$x_ccdred.e long+


so I have a better idea of what version you are using?

If this is some new form of 64-bit bug then it might be that using a 32-bit binary would fix the problem. I tested against the distributed binaries to make rule out any recent bug fix in the development system. If you'd like to try the same binaries I used I've posted these at

ftp://iraf.noao.edu/pub/x_ccdred.e.LINUX
ftp://iraf.noao.edu/pub/x_ccdred.e.LINUX64

Try the 64-bit binary first, if that still fails try the 32-bit. If both fail then it may be related to the newer kernel (more likely, newer gcc) but I'd have to hunt around to find a similar test system.

Vallery Stanishev wrote on May 01, 2012

For me both the 64bit and the 32-bit binaries did not work. My problems still persist. My original binary was:


-rwxr-xr-x. 1 vall vall 9035520 May 24 2012 x_ccdred.e.orig

Mike Fitzpatrick wrote on May 01, 2012

That appears to be the last released binary, and the same one that worked for me. If you're up to trying one more binary, see

ftp://iraf.noao.edu/pub/x_ccdred.e.LINUX64-STATIC

This is statically linked and if it works may point to something in a newer gcc/glibc (could you post those versions as well) release.

Vallery Stanishev wrote on May 01, 2012

fitz

That appears to be the last released binary, and the same one that worked for me. If you're up to trying one more binary, see

ftp://iraf.noao.edu/pub/x_ccdred.e.LINUX64-STATIC

This is statically linked and if it works may point to something in a newer gcc/glibc (could you post those versions as well) release.


It did not work with that binary either. Here are my gcc/glibc/kernel versions:

gcc-4.7.2-8.fc18.x86_64
glibc-2.16-28.fc18.x86_64
glibc-2.16-28.fc18.i686
kernel-3.8.1-201.fc18.x86_64

Mike Fitzpatrick wrote on May 01, 2012

Still no luck: I built a new FC18 machine with all the latest patches applied, installed the most recent v2.16 and still couldn't reproduce the problem.

At this point I'd suggest a reinstall of IRAF in case the files were somehow corrupted. If you have another FC18 machine try it there, if it works then it isn't impossible it could be a hardware issue such as a bad memory card. Are there any suspicious system log messages? Since multiple users are affected hardware problems are unlikely the cause, if anyone wants to send me login info so I can reproduce the problem on their machine I'd be happy to follow up.

Vallery Stanishev wrote on May 01, 2012

fitz

Still no luck: I built a new FC18 machine with all the latest patches applied, installed the most recent v2.16 and still couldn't reproduce the problem.

At this point I'd suggest a reinstall of IRAF in case the files were somehow corrupted. If you have another FC18 machine try it there, if it works then it isn't impossible it could be a hardware issue such as a bad memory card. Are there any suspicious system log messages? Since multiple users are affected hardware problems are unlikely the cause, if anyone wants to send me login info so I can reproduce the problem on their machine I'd be happy to follow up.


No luck here too. I re-installed IRAF from scratch and I still get the very same errors. What bothers me most is that a command as simple as 'imdel' will give Segmantation error!

I would be happy to provide you with login details, but I cannot do it now. Maybe in two weeks time.

Mike Fitzpatrick wrote on May 01, 2012

Re: the imdel segfault:

There used to be a bug caused by the 'cache' environment variable being set to "/tmp" but this should be fixed in the version you have. In any case, do a "cl> show cache" and make sure 1) it isn't set to /tmp and 2) that the path has a trailing '/' on it. Do something like "cl> reset cache = uparm$" if you need to reset a /tmp value.

When you're ready with the login info just send it to admin-at-iraf.net and I'll have a look. Until then, if you can get a core dump from the error (e.g. do a "limit coredumpsize unlimited" in a C-shell before start IRAF) you can post that to ftp://iraf.noao.edu/pub and I'll see what that tells me on my machine.

Vallery Stanishev wrote on May 01, 2012

That was it! 'mkiraf' has set cache="/tmp//". I changed it back to "/iraf/cache/" and now everything works again. Thanks for the help.

Last post on May 01, 2012