Memory corrupted 2
hmessias wrote on May 21, 2008
Dear all,
I have using imcombine on macOSX and it works fine. Now I am doing the same but on Ubuntu. Sure, it is a architecture problem but even after consulting these forums the problem subsists.
So, the situation is as follows: I am trying to combine a group of stamps (60x60 pix) each time I run the script. Each sample/group has a different amount of stamps. When I combine the larger ones I run into problems. IRAF gives me this error message:
PANIC in `/iraf/iraf/bin.linux/x_images.e': Memory has been corrupted
Before, as I told before, I was working in a macOSX but also with a different image from which I cut the stamps. Passing to Ubuntu I kept the image and it still worked fine (even with the large samples). It was only when I changed the initial image that the problems started to appear.
The images are from the same sky coverage but with different telescopes. Each sample has the same coordinates no matter the initial image. The stamps are still 60x60pix although the resolution in each image is different.
I came across this post in the "General IRAF" forum:
http://iraf.net/phpBB2/viewtopic.php?t=86285&highlight=memory
I noticed that the hlib$cl.csh script already takes this probable problem into account as you can see (in my case $IRAFARCH = linux):
if ("$IRAFARCH" == "redhat" || \
"$IRAFARCH" == "linux" || \
"$IRAFARCH" == "linuxppc") then
limit stacksize unlimited
endif
Suspecting that it might not be working, I wrote in the beginning of the script:
#because of memory corrupted
limit stacksize unlimited
It did not work as well, so I proceeded with the .bashrc writing:
ulimit -s unlimited
Again, it didn't work.
What might be causing this "memory corrupted" problem? I even passed through "kvis" (i.e., resaved as fits) the initial fits image from which I cut the stamps.
One difference between the two is the coordinates type. One is NCP (the first and gives no error message) and the other is SIN (gives error). But I don't think this is the problem as with "hedit" I changed the coordinates type to TAN as IRAF asks to.
Thank you in advance!
Cheers!
Hugo
I have using imcombine on macOSX and it works fine. Now I am doing the same but on Ubuntu. Sure, it is a architecture problem but even after consulting these forums the problem subsists.
So, the situation is as follows: I am trying to combine a group of stamps (60x60 pix) each time I run the script. Each sample/group has a different amount of stamps. When I combine the larger ones I run into problems. IRAF gives me this error message:
PANIC in `/iraf/iraf/bin.linux/x_images.e': Memory has been corrupted
Before, as I told before, I was working in a macOSX but also with a different image from which I cut the stamps. Passing to Ubuntu I kept the image and it still worked fine (even with the large samples). It was only when I changed the initial image that the problems started to appear.
The images are from the same sky coverage but with different telescopes. Each sample has the same coordinates no matter the initial image. The stamps are still 60x60pix although the resolution in each image is different.
I came across this post in the "General IRAF" forum:
http://iraf.net/phpBB2/viewtopic.php?t=86285&highlight=memory
I noticed that the hlib$cl.csh script already takes this probable problem into account as you can see (in my case $IRAFARCH = linux):
if ("$IRAFARCH" == "redhat" || \
"$IRAFARCH" == "linux" || \
"$IRAFARCH" == "linuxppc") then
limit stacksize unlimited
endif
Suspecting that it might not be working, I wrote in the beginning of the script:
#because of memory corrupted
limit stacksize unlimited
It did not work as well, so I proceeded with the .bashrc writing:
ulimit -s unlimited
Again, it didn't work.
What might be causing this "memory corrupted" problem? I even passed through "kvis" (i.e., resaved as fits) the initial fits image from which I cut the stamps.
One difference between the two is the coordinates type. One is NCP (the first and gives no error message) and the other is SIN (gives error). But I don't think this is the problem as with "hedit" I changed the coordinates type to TAN as IRAF asks to.
Thank you in advance!
Cheers!
Hugo
Mike Fitzpatrick wrote on May 21, 2008
The problem appears to be related to the data but there's really not enough information to comment on what the problem is. A 'memory corruption' usually indicates a bug in the task/system but we'd need to be able to reproduce it in order to fix it.
Could you please post the parameters you were using (e.g. "cl> dpar imcombine") as well as a URL to the image? You could also upload the image via anonftp to ftp://iraf.noao.edu/pub
Cheers,
-Mike
Could you please post the parameters you were using (e.g. "cl> dpar imcombine") as well as a URL to the image? You could also upload the image via anonftp to ftp://iraf.noao.edu/pub
Cheers,
-Mike
hmessias wrote on May 21, 2008
Hi,
The data is the first data release from 1.4GHz VLA observations by Nean Miller et al. 2008 (http://arxiv.org/abs/0804.2375). The image can be obtained on this site:
http://taltos.pha.jhu.edu/~nmiller/vlaecdfs_main.html
In the "Data" link you can see how to obtain the image.
The parameters I used were:
imcombine in=@stamp.in weight=@weight.list combine=sum reject=sigclip mclip=no nkeep=xxx lsigma=3 hsig=3 output=sum_rej nrejmas=sum_rej_nrej expmask=sum_rej_expm sigmas=sum_rej_sig
stamp.in - file with a list of stamps to enter in the image combination;
weight.list - list of weights of each stamp;
nkeep - 80% of the total number of stamps that entered initially;
Each stamp is a 60x60pix of particular regions of the initial image, in this case, the VLA Extended Chandra Deep Field South mosaic image. The problem appears when the total number of stamps is bigger than 260 as I believe.
I have cut a number of stamps already with miriad. Do you want them or you rather cut randomly yourself to try it out?
Cheers!
Hugo
The data is the first data release from 1.4GHz VLA observations by Nean Miller et al. 2008 (http://arxiv.org/abs/0804.2375). The image can be obtained on this site:
http://taltos.pha.jhu.edu/~nmiller/vlaecdfs_main.html
In the "Data" link you can see how to obtain the image.
The parameters I used were:
imcombine in=@stamp.in weight=@weight.list combine=sum reject=sigclip mclip=no nkeep=xxx lsigma=3 hsig=3 output=sum_rej nrejmas=sum_rej_nrej expmask=sum_rej_expm sigmas=sum_rej_sig
stamp.in - file with a list of stamps to enter in the image combination;
weight.list - list of weights of each stamp;
nkeep - 80% of the total number of stamps that entered initially;
Each stamp is a 60x60pix of particular regions of the initial image, in this case, the VLA Extended Chandra Deep Field South mosaic image. The problem appears when the total number of stamps is bigger than 260 as I believe.
I have cut a number of stamps already with miriad. Do you want them or you rather cut randomly yourself to try it out?
Cheers!
Hugo
hmessias wrote on May 21, 2008
Hi again,
Does it help changing the environment variable imcombine_maxmemory?
I tried with different values with no results but I don't know how much do I have to change the initial 250000000 value in order to get it working.
Cheers!
Hugo
Does it help changing the environment variable imcombine_maxmemory?
I tried with different values with no results but I don't know how much do I have to change the initial 250000000 value in order to get it working.
Cheers!
Hugo
Francisco Valdes wrote on May 21, 2008
Hi,
I'm afraid you will have to find a workaround for this because the resources needed to reproduce and then debug this are not available here. You could combine in smaller groups and then combine the groups. The maxmemory feature can help with some problems but not all. It is not likely to help with the memory corruption error since mostly it helps when the error is "out of memory". You could try it and change the value down by a factor of 10 and then 100.
If we get more reports, particularly with something that is easy to reproduce, this will get fixed in a future release.
I'm sorry for the problem and the less than satisfactory answer.
Yours,
Frank Valdes
I'm afraid you will have to find a workaround for this because the resources needed to reproduce and then debug this are not available here. You could combine in smaller groups and then combine the groups. The maxmemory feature can help with some problems but not all. It is not likely to help with the memory corruption error since mostly it helps when the error is "out of memory". You could try it and change the value down by a factor of 10 and then 100.
If we get more reports, particularly with something that is easy to reproduce, this will get fixed in a future release.
I'm sorry for the problem and the less than satisfactory answer.
Yours,
Frank Valdes
hmessias wrote on May 21, 2008
Hi,
Yes, that seems to be the best way to skirt this situation.
Thank you though! If I have any improvement on this I'll leave it here.
Cheers!
Hugo
Yes, that seems to be the best way to skirt this situation.
Thank you though! If I have any improvement on this I'll leave it here.
Cheers!
Hugo
Last post on May 21, 2008