Possible bug in craverage: floating point overflow error
emma wrote on Jul 06, 2011
Hello,
When I run the following command on my linux machine (a "more /etc/redhat-release" gives "Red Hat Enterprise Linux Client release 5.2 (Tikanga)") I have no problems:
where readnoise and gain are variables that have values as defined by the read noise and gain keywords in the header of the input file.
However, when I run the same command on my 10.5.8 macbook I get
The problem is related to the fact that the image defined by the crmask parameter exists and according to the help is "used to exclude data pixels from the calculations". I believe that this is what is causing the "floating point overflow" error, because if I change the crmask parameter to a file that doesn't exist, craverage successfully completes on both linux and mac and creates an output file as defined by the crmask parameter that contains the locations of the CRs.
Both the linux and mac are using IRAF v2.14.1. I initialised my uparm directory before running craverage in both cases.
I can provide you with an example fits file that can reproduce this problem if required :)
Many thanks,
Emma :)
When I run the following command on my linux machine (a "more /etc/redhat-release" gives "Red Hat Enterprise Linux Client release 5.2 (Tikanga)") I have no problems:
craverage (input="input.fits[SCI,1]", output="" , crmask="input.fits[DQ,1]", average="", sigma="", navg=9, nrej=48, nbkg=11, nsig=25, var0=(readnoise**2), var1=gain, var2=0, crval=8, lcrsig=5, hcrsig=10, crgrow=1, objval=0, lobjsig=10, hobjsig=10, objgrow=0)
where readnoise and gain are variables that have values as defined by the read noise and gain keywords in the header of the input file.
However, when I run the same command on my 10.5.8 macbook I get
ERROR: floating point overflow
The problem is related to the fact that the image defined by the crmask parameter exists and according to the help is "used to exclude data pixels from the calculations". I believe that this is what is causing the "floating point overflow" error, because if I change the crmask parameter to a file that doesn't exist, craverage successfully completes on both linux and mac and creates an output file as defined by the crmask parameter that contains the locations of the CRs.
Both the linux and mac are using IRAF v2.14.1. I initialised my uparm directory before running craverage in both cases.
I can provide you with an example fits file that can reproduce this problem if required :)
Many thanks,
Emma :)
Mike Fitzpatrick wrote on Jul 06, 2011
Since it's a data-dependent problem then example FITS files would be required to reproduce it, otherwise there are no known issues with the task. Please upload the files to the anonftp at ftp://iraf.noao.edu/pub and I'll have a look.
emma wrote on Jul 06, 2011
Awesomely quick response time :D
The file has been uploaded ... I called it test_craverage.fits.gz
Thanks Mike! :)
The file has been uploaded ... I called it test_craverage.fits.gz
Thanks Mike! :)
emma wrote on Jul 06, 2011
Sorry Mike, I uploaded the wrong file ... I'm trying to upload a new one (with a different name) but it tells me
I am using the same commands as before ...
553 Could not create file.
I am using the same commands as before ...
Mike Fitzpatrick wrote on Jul 06, 2011
You sure that's the right image? There is only one MEF extension, and it doesn't have a extname/extver. The error message might be bogus, but the problem may just be that there is no SCI or DQ extension in the image.
emma wrote on Jul 06, 2011
Ok, I got it, it's called test_craverage_new.fits this time :)
Mike Fitzpatrick wrote on Jul 06, 2011
I'm able to reproduce the problem, but it doesn't look trivial. Since this is Frank's task and he's more familiar with the algorithm I've asked him to take over, unfortunately he is away until next week (when I then leave). Somebody will get back to you, but post back if you don't hear anything. Thanks for the report.
emma wrote on Jul 06, 2011
That's great, thank you! :)
Francisco Valdes wrote on Jul 06, 2011
Hi Emma,
I took a look at your example and I have not yet reproduced it. I am waiting to hear from Mike about how he reproduced the error.
But I wanted to also reply to comment on your usage. Craverage is primarily intended for direct images. It is probably not very appropriate for your spectral data. The cosmic rays outside the orders can be easily identified but they don't really have any impact on the later spectral processing. Within the orders a 2D CR detection method, such as craverage, is not a good choice. In fact in my running of this I see the central ridge line of the left 3 orders are totally being flagged as cosmic rays. As a hint:
For spectral data it is better to detect CRs against the spectral profile if you are extracting to 1D using apextract. If you are doing science on the 2D slit data then it is pretty difficult without a cosmic ray split set of exposures.
So I suggest you think a bit more about how to handle the cosmic rays further downstream in your reductions.
Yours,
Frank Valdes
I took a look at your example and I have not yet reproduced it. I am waiting to hear from Mike about how he reproduced the error.
But I wanted to also reply to comment on your usage. Craverage is primarily intended for direct images. It is probably not very appropriate for your spectral data. The cosmic rays outside the orders can be easily identified but they don't really have any impact on the later spectral processing. Within the orders a 2D CR detection method, such as craverage, is not a good choice. In fact in my running of this I see the central ridge line of the left 3 orders are totally being flagged as cosmic rays. As a hint:
display test_craverage_new.fits[SCI,1] 2 over=test_craverage_new.fits[DQ,1]For spectral data it is better to detect CRs against the spectral profile if you are extracting to 1D using apextract. If you are doing science on the 2D slit data then it is pretty difficult without a cosmic ray split set of exposures.
So I suggest you think a bit more about how to handle the cosmic rays further downstream in your reductions.
Yours,
Frank Valdes
Last post on Jul 06, 2011