how to use apall task to clean echelle data
Guest wrote on Feb 13, 2009
Hello everybody,
I want to extract echelle data. I normalized my flatfield with the task âapnormalizeâ and divide my object frame with the normalized flatfield. I find there are four long bright patches along the dispersion direction affecting the first 3 apertures of my object. The sky background was then subtracted with âapscatterâ. After extraction with âapallâ, there are wild oscillations in parts of the spectra in the 1st 3 apertures. This is probably due to the bright patches. Running âapallâ on unflatfielded data does not have that effect. My flats were combined using âcrrejectâ algorithm. How can I remove those bright patches? Below are extraction parameters for âapallâ:
Please help. :cry:
Siti
I want to extract echelle data. I normalized my flatfield with the task âapnormalizeâ and divide my object frame with the normalized flatfield. I find there are four long bright patches along the dispersion direction affecting the first 3 apertures of my object. The sky background was then subtracted with âapscatterâ. After extraction with âapallâ, there are wild oscillations in parts of the spectra in the 1st 3 apertures. This is probably due to the bright patches. Running âapallâ on unflatfielded data does not have that effect. My flats were combined using âcrrejectâ algorithm. How can I remove those bright patches? Below are extraction parameters for âapallâ:
# EXTRACTION PARAMETERS
(backgro= none) Background to subtract
(skybox = 1) Box car smoothing length for sky
(weights= variance) Extraction weights (none|variance)
(pfit = fit1d) Profile fitting type (fit1d|fit2d)
(clean = yes) Detect and replace bad pixels?
(saturat= INDEF) Saturation level
(readnoi= 8) Read out noise sigma (photons)
(gain = 1.7) Photon gain (photons/data number)
(lsigma = 4.) Lower rejection threshold
(usigma = 4.) Upper rejection threshold
(nsubaps= 1) Number of subapertures per aperture
(mode = ql)
Please help. :cry:
Siti
Francisco Valdes wrote on Feb 13, 2009
The oscillations probably come from the profile modeling. Try not to use the "weighting" and "cleaning" algorithms which can sometimes cause this behavior. Other than this hint there is nothing specific I can tell you.
Yours,
Frank Valdes
Yours,
Frank Valdes
Guest wrote on Feb 13, 2009
Hi Frank,
I did a quick extraction of the spectra without background subtraction (which is a constant anyway). In all I set the saturation level = 29412 ADU (50,000 e). Applying different combinations of cleaning and weighting algorithms give very different statistics except 2nd and 3rd lines. See below. Without the cleaning, there will be lots of cosmic ray emissions in my data. :?
-----------------------------------------MEAN STDDEV MIN MAX
Clean= Yes,Weight= Var, Flat=No 1315. 647.9 48.99 31784.
Clean= Yes,Weight= Var, Flat=Yes 1347. 721. 7.024 35320.
Clean= Yes,Weight=No, Flat=Yes 1347. 721. 7.024 35320.
Clean= No,Weight=Var, Flat=Yes 1405. 1871. -6326. 268827.
Clean= No,Weight=No, Flat=Yes 1405. 888.8 -276.9 51932.
However, the wild oscillations in the first few apertures exist only in all flatfielded data. This led me to believe that parts of the flatfield may be defective despite already fixed what I thought are the bad columns in the flatfield before normalization. (I used âccdprocâ and âfixpixâ). From your experience as spectroscopist, could the flatfield be the cause? If so, probably I donât have a choice but to simply remove the bad parts of the spectra.
Another question, what is the best way to remove remaining cosmic rays after extraction? I am not done yet but âscombineâ looks like the most likely option to me.
Thanks for your feedback.
:) Siti
I did a quick extraction of the spectra without background subtraction (which is a constant anyway). In all I set the saturation level = 29412 ADU (50,000 e). Applying different combinations of cleaning and weighting algorithms give very different statistics except 2nd and 3rd lines. See below. Without the cleaning, there will be lots of cosmic ray emissions in my data. :?
-----------------------------------------MEAN STDDEV MIN MAX
Clean= Yes,Weight= Var, Flat=No 1315. 647.9 48.99 31784.
Clean= Yes,Weight= Var, Flat=Yes 1347. 721. 7.024 35320.
Clean= Yes,Weight=No, Flat=Yes 1347. 721. 7.024 35320.
Clean= No,Weight=Var, Flat=Yes 1405. 1871. -6326. 268827.
Clean= No,Weight=No, Flat=Yes 1405. 888.8 -276.9 51932.
However, the wild oscillations in the first few apertures exist only in all flatfielded data. This led me to believe that parts of the flatfield may be defective despite already fixed what I thought are the bad columns in the flatfield before normalization. (I used âccdprocâ and âfixpixâ). From your experience as spectroscopist, could the flatfield be the cause? If so, probably I donât have a choice but to simply remove the bad parts of the spectra.
Another question, what is the best way to remove remaining cosmic rays after extraction? I am not done yet but âscombineâ looks like the most likely option to me.
Thanks for your feedback.
:) Siti
Francisco Valdes wrote on Feb 13, 2009
Dear Siti,
What I didn't make clear is that the processing of the flat field with apnorm involves the same profile modeling used for cleaning and weighting which is subject to producing oscillations. So you should return to figuring out how best to process the flat fields, as you seem to be realizing. You can look at the apnorm parameters and make sure the weighting and cleaning are turned off. You might also consider using apflatten instead. It is always confusing to people whether they should use apflatten or apnorm. The difference is in whether the shape of the profile across the dispersion is removed, i.e. normalized. I suspect you will do better with using apflatten to remove the dispersion shape of the flat field (and also mask out the regions outside the profiles to avoid division by very small values).
As for cleaning, it aways depends on what data you have at your disposal, particularly if you have multiple exposures. With single exposures the cleaning during aperture extraction is the best followed by cleaning in 1D with something like lineclean. With multiple exposures the choices are to combine with rejection either in 2D (say with imcombine) or in 1D with scombine. I usually tell people to try it both ways. The main factor is whether their is any flexture (shift of the profiles) in the 2D data.
Yours,
Frank
What I didn't make clear is that the processing of the flat field with apnorm involves the same profile modeling used for cleaning and weighting which is subject to producing oscillations. So you should return to figuring out how best to process the flat fields, as you seem to be realizing. You can look at the apnorm parameters and make sure the weighting and cleaning are turned off. You might also consider using apflatten instead. It is always confusing to people whether they should use apflatten or apnorm. The difference is in whether the shape of the profile across the dispersion is removed, i.e. normalized. I suspect you will do better with using apflatten to remove the dispersion shape of the flat field (and also mask out the regions outside the profiles to avoid division by very small values).
As for cleaning, it aways depends on what data you have at your disposal, particularly if you have multiple exposures. With single exposures the cleaning during aperture extraction is the best followed by cleaning in 1D with something like lineclean. With multiple exposures the choices are to combine with rejection either in 2D (say with imcombine) or in 1D with scombine. I usually tell people to try it both ways. The main factor is whether their is any flexture (shift of the profiles) in the 2D data.
Yours,
Frank
Guest wrote on Feb 13, 2009
Dear Frank,
I mistakenly thought you were talking about âapallâ!
I did that but didnât work.
Using apflatten solves my problem. :D But I didnât know when to use apflatten or apnorm.
I have 2 or 3 exposures for each object. I will try all the suggestions.
Thanks for your ideas.
Siti
What I didn't make clear is that the processing of the flat field with apnorm involves the same profile modeling used for cleaning and weighting which is subject to producing oscillations.
I mistakenly thought you were talking about âapallâ!
You can look at the apnorm parameters and make sure the weighting and cleaning are turned off.
I did that but didnât work.
You might also consider using apflatten instead. It is always confusing to people whether they should use apflatten or apnorm.
Using apflatten solves my problem. :D But I didnât know when to use apflatten or apnorm.
As for cleaning, it aways depends on what data you have at your disposal, particularly if you have multiple exposures.
I have 2 or 3 exposures for each object. I will try all the suggestions.
Thanks for your ideas.
Siti
Last post on Feb 13, 2009