SOS!!!! - problem with response task
Danilenko Andrey wrote on Jul 03, 2006
Hello everybody,
Please, help.
I have some problem with response task. This problem has already
worry me. :(
The heart of the problem in few words:
I set parameters for response:
PACKAGE = longslit
TASK = response
calibrat= FlatS.fits Longslit calibration images
normaliz= FlatS.fits Normalization spectrum images
response= nFlatS.fits Response function images
(interac= yes) Fit normalization spectrum interactively?
(thresho= INDEF) Response threshold
(sample = *) Sample of points to use in fit
(naverag= 1) Number of points in sample averaging
(functio= spline3) Fitting function
(order = 6) Order of fitting function
(low_rej= 3.) Low rejection in sigma of fit
(high_re= 3.) High rejection in sigma of fit
(niterat= 1) Number of rejection iterations
(grow = 0.) Rejection growing radius
(graphic= stdgraph) Graphics output device
(cursor = ) Graphics cursor input
(mode = ql)
The input calibration image FlatS.fits have such statistics:
lo> imstat FlatS.fits
# IMAGE NPIX MEAN STDDEV MIN MAX
FlatS.fits 2095100 1586. 2294. 1. 9448.
It is shown that min count level is 1 for FlatS.fits and so min count level is such as it is needed
for flat field image.
Then I run response. I get the output image nFlatS.fits. The output image nFlatS.fits have such
statistics:
lo> imstat nFlatS.fits
# IMAGE NPIX MEAN STDDEV MIN MAX
nFlatS.fits 2095100 0.6434 3.051 -34.6 235.9
It is shown that min count level is -34.6 but it is not right for flat field image.
Why it is happened ? And how can I correct this mistake ?
thanks for your attention and sorry for my bad english.
Please, help.
I have some problem with response task. This problem has already
worry me. :(
The heart of the problem in few words:
I set parameters for response:
PACKAGE = longslit
TASK = response
calibrat= FlatS.fits Longslit calibration images
normaliz= FlatS.fits Normalization spectrum images
response= nFlatS.fits Response function images
(interac= yes) Fit normalization spectrum interactively?
(thresho= INDEF) Response threshold
(sample = *) Sample of points to use in fit
(naverag= 1) Number of points in sample averaging
(functio= spline3) Fitting function
(order = 6) Order of fitting function
(low_rej= 3.) Low rejection in sigma of fit
(high_re= 3.) High rejection in sigma of fit
(niterat= 1) Number of rejection iterations
(grow = 0.) Rejection growing radius
(graphic= stdgraph) Graphics output device
(cursor = ) Graphics cursor input
(mode = ql)
The input calibration image FlatS.fits have such statistics:
lo> imstat FlatS.fits
# IMAGE NPIX MEAN STDDEV MIN MAX
FlatS.fits 2095100 1586. 2294. 1. 9448.
It is shown that min count level is 1 for FlatS.fits and so min count level is such as it is needed
for flat field image.
Then I run response. I get the output image nFlatS.fits. The output image nFlatS.fits have such
statistics:
lo> imstat nFlatS.fits
# IMAGE NPIX MEAN STDDEV MIN MAX
nFlatS.fits 2095100 0.6434 3.051 -34.6 235.9
It is shown that min count level is -34.6 but it is not right for flat field image.
Why it is happened ? And how can I correct this mistake ?
thanks for your attention and sorry for my bad english.
Francisco Valdes wrote on Jul 03, 2006
Hello Danila,
Your flat field must have low response (low counts) at some wavelengths. The only task that does anything special for flat fields is CCDPROC which will limit the values of the processed flat field to be greater than some value set by a parameter and usually defaults to 1.
RESPONSE does not do this, though you could argue that it should. What I believe is happening is that when the average flat field spectrum is extracted it
has values near zero. The process that fits this spectrum to smooth it, in your case a cubic spline of 6 pieces unless you changed the values interactively, will normally including some "wiggles" as it tries to approximate the shape of the spectrum. These wiggles can cause the fit to go negative where the signal is very low.
My suggestion is that you apply IMREPLACE after RESPONSE. This task lets you replace all values below some amount, such as 1, by a constant value, such as 1.
I hope this explains your problem and gives a good solution.
Yours,
Frank Valdes
Your flat field must have low response (low counts) at some wavelengths. The only task that does anything special for flat fields is CCDPROC which will limit the values of the processed flat field to be greater than some value set by a parameter and usually defaults to 1.
RESPONSE does not do this, though you could argue that it should. What I believe is happening is that when the average flat field spectrum is extracted it
has values near zero. The process that fits this spectrum to smooth it, in your case a cubic spline of 6 pieces unless you changed the values interactively, will normally including some "wiggles" as it tries to approximate the shape of the spectrum. These wiggles can cause the fit to go negative where the signal is very low.
My suggestion is that you apply IMREPLACE after RESPONSE. This task lets you replace all values below some amount, such as 1, by a constant value, such as 1.
I hope this explains your problem and gives a good solution.
Yours,
Frank Valdes
Danilenko Andrey wrote on Jul 03, 2006
Hello Frank,
Thank your for usefull advice. I will try to run IMREPLACE.
But I still not quate understand ...
You write: "... where the signal is very low". But I note that "min count level is 1" for input flat field image and so signal is't very low.
Yours,
Andrey Danilenko
Thank your for usefull advice. I will try to run IMREPLACE.
But I still not quate understand ...
You write: "... where the signal is very low". But I note that "min count level is 1" for input flat field image and so signal is't very low.
Yours,
Andrey Danilenko
Francisco Valdes wrote on Jul 03, 2006
Hello Andrey,
I am used to seeing flat field data that has not been normalized but is in the original CCD counts. These are typically 1000-10000 in the instrument units. A value of 1 digital count is very low in this case. However, if you do normalize by something like the mean value then you are correct that 1 is not a low value but was originally the normalization value. So what I said depends on units.
Is this why you say 1 is not low? If you have normalized than having a statistic with a minimum of -3 is significantly wrong I think.
Frank
I am used to seeing flat field data that has not been normalized but is in the original CCD counts. These are typically 1000-10000 in the instrument units. A value of 1 digital count is very low in this case. However, if you do normalize by something like the mean value then you are correct that 1 is not a low value but was originally the normalization value. So what I said depends on units.
Is this why you say 1 is not low? If you have normalized than having a statistic with a minimum of -3 is significantly wrong I think.
Frank
Danilenko Andrey wrote on Jul 03, 2006
Frank,
You are right that my
flat field data (for wich I apply response task ) has not been normalized. So you say that in this case I can get output file with min count level below 1 after response task has been applyed. As I understand if I do normalize my flat field by mean value before I run response so I get correct output flat field. Is I correct?
How can I do normalize?
Andrey
You are right that my
flat field data (for wich I apply response task ) has not been normalized. So you say that in this case I can get output file with min count level below 1 after response task has been applyed. As I understand if I do normalize my flat field by mean value before I run response so I get correct output flat field. Is I correct?
How can I do normalize?
Andrey
Francisco Valdes wrote on Jul 03, 2006
Audrey,
No you should not normalize. You should use IMREPLACE as I told you earlier. All that normalizing will do is change the units. It will not change the behavior of getting negative values.
The last message was only to tell you that a small value really does mean that you have almost no signal and the results will be poor. Good flat field values should be more than 100 digital counts. There is nothing you can do about that without more data.
Frank
No you should not normalize. You should use IMREPLACE as I told you earlier. All that normalizing will do is change the units. It will not change the behavior of getting negative values.
The last message was only to tell you that a small value really does mean that you have almost no signal and the results will be poor. Good flat field values should be more than 100 digital counts. There is nothing you can do about that without more data.
Frank
Danilenko Andrey wrote on Jul 03, 2006
Frank,
Ok, I understand.
So when I do ccd reduction of spectral data It is always needed to apply IMREPLACE after RESPONSE. Is it right?
Andrey
Ok, I understand.
So when I do ccd reduction of spectral data It is always needed to apply IMREPLACE after RESPONSE. Is it right?
Andrey
Francisco Valdes wrote on Jul 03, 2006
Only if the output of RESPONSE contains values less than or equal to zero. You only get low signal when your spectrum has wavelengths where the CCD is not sensitive or if there is a filter that cuts off before the spectrum goes off the edge of the CCD.
If you want a simple answer then it is ok to aways use IMREPLACE after response even if it is not needed.
Frank
If you want a simple answer then it is ok to aways use IMREPLACE after response even if it is not needed.
Frank
Danilenko Andrey wrote on Jul 03, 2006
Frank,
I really understand. Thank you very much for useful discussion!
I must to come out - there is late evening in Russia. I will follow your advice tomorrow and reply about result.
with the best wishes,
Andrey
I really understand. Thank you very much for useful discussion!
I must to come out - there is late evening in Russia. I will follow your advice tomorrow and reply about result.
with the best wishes,
Andrey
Danilenko Andrey wrote on Jul 03, 2006
Hello Frank,
I just have followed your advise to apply IMREPLACE to output of RESPONSE. At first sight I get a good result.
Thank you very much Frank!
with hight respect and the best wishes,
Andrey
I just have followed your advise to apply IMREPLACE to output of RESPONSE. At first sight I get a good result.
Thank you very much Frank!
with hight respect and the best wishes,
Andrey
Last post on Jul 03, 2006