problem with gsflat of gmos gemini package
jorge diaz wrote on Jun 10, 2010
Hi all:
I have a problem with gsflat task from gmos gemini package. It gives the following error msg when i try to reduce spectroscopic flats of standar star files taken in longslit mode:
GIREDUCE exit status: good.
----------------------------------------------------------------------------
GSREDUCE: Multiplying by the gains, using GGAIN
GSREDUCE: Calling GMOSAIC
GLOGINIT ERROR: 101 Unable to access file 'gmos.log'..
GSREDUCE: returned from GMOSAIC.
Warning: cannot rename `tmpimage26556o.fits' -> `gsgs1031075.fits'
GSREDUCE: Calling GSAPPWAVE
--------------------------------------------------------------------------------
and the process continue giving errors.
I think that the problem is maybe about gmosaic, i didn't have problem with the science frames, only with standar star data, and I'm not how to fix it.
I also have this problem using gsreduce with the remain calibration data related with standar stars, but i can deal deactivating the param related with gmosaic, option that it is unavailable in gsflat.
well i hope that someone can help me.
Best.
I have a problem with gsflat task from gmos gemini package. It gives the following error msg when i try to reduce spectroscopic flats of standar star files taken in longslit mode:
GIREDUCE exit status: good.
----------------------------------------------------------------------------
GSREDUCE: Multiplying by the gains, using GGAIN
GSREDUCE: Calling GMOSAIC
GLOGINIT ERROR: 101 Unable to access file 'gmos.log'..
GSREDUCE: returned from GMOSAIC.
Warning: cannot rename `tmpimage26556o.fits' -> `gsgs1031075.fits'
GSREDUCE: Calling GSAPPWAVE
--------------------------------------------------------------------------------
and the process continue giving errors.
I think that the problem is maybe about gmosaic, i didn't have problem with the science frames, only with standar star data, and I'm not how to fix it.
I also have this problem using gsreduce with the remain calibration data related with standar stars, but i can deal deactivating the param related with gmosaic, option that it is unavailable in gsflat.
well i hope that someone can help me.
Best.
emma wrote on Jun 10, 2010
Hello Jorge,
Do you have write permissions in the current working directory? Did you reduce your science data in a different directory to your standard star data? Can you supply your call to gsflat and the list of parameters you used, please?
Many thanks,
Emma :)
Do you have write permissions in the current working directory? Did you reduce your science data in a different directory to your standard star data? Can you supply your call to gsflat and the list of parameters you used, please?
Many thanks,
Emma :)
jorge diaz wrote on Jun 10, 2010
Hi Emma:
I have write permission to write in the directory, I reduced the science frames in other directory, but i just made a mkiraf dir and none special procedure in both directories. The param list of gsflat is the following:
inflats,s,a,"gs1031075",,,"Input flatfields"
specflat,s,a,"fs1031075",,,"Output normalized flat (MEF)"
fl_answer,b,a,yes,,,"Continue interactive fitting?"
fl_slitcorr,b,h,no,,,"Correct output for Illumination/Slit-Function"
slitfunc,s,h,"",,,"Slit Function (MEF output of gsslitfunc)"
fl_keep,b,h,no,,,"Keep imcombined flat?"
combflat,s,h,"",,,"Filename for imcombined flat"
fl_over,b,h,yes,,,"Subtract overscan level"
fl_trim,b,h,yes,,,"Trim off overscan region"
fl_bias,b,h,yes,,,"Subtract bias image"
fl_dark,b,h,no,,,"Subtract (scaled) dark image"
fl_fixpix,b,h,no,,,"Interpolate across chip gaps"
fl_vardq,b,h,no,,,"Create variance and data quality frames"
bias,s,h,"gs1125_bias",,,"Bias image"
dark,s,h,"",,,"Dark image"
key_exptime,s,h,"EXPTIME",,,"Exposure time header keyword"
key_biassec,s,h,"BIASSEC",,,"Header keyword for overscan strip image section"
key_datasec,s,h,"DATASEC",,,"Header keyword for data section (excludes the overscan)"
rawpath,s,h,"",,,"GPREPARE: Path for input raw images"
sci_ext,s,h,"SCI",,,"Name of science extension"
var_ext,s,h,"VAR",,,"Name of variance extension"
dq_ext,s,h,"DQ",,,"Name of data quality extension"
key_mdf,s,h,"MASKNAME",,,"Header keyword for the MDF"
mdffile,s,h,"",,,"MDF to use if keyword not found"
mdfdir,s,h,"gmos$data/",,,"MDF database directory"
bpm,s,h,"",,,"Name of bad pixel mask file or image"
gaindb,s,h,"default",,,"Database with gain data"
gratingdb,s,h,"gmos$data/GMOSgratings.dat",,,"Gratings database file"
filterdb,s,h,"gmos$data/GMOSfilters.dat",,,"Filters database file"
bpmfile,s,h,"gmos$data/chipgaps.dat",,,"Info on location of chip gaps"
refimage,s,h,"",,,"Reference image for slit positions"
sat,r,h,65000.,,,"Saturation level in raw images"
xoffset,r,h,INDEF,,,"X offset in wavelength [nm]"
yoffset,r,h,INDEF,,,"Y offset in unbinned pixels"
yadd,r,h,0.,,,"Additional pixels to add to each end of MOS slitlet lengths"
fl_usegrad,b,h,no,,,"Use gradient method to find MOS slits"
fl_emis,b,h,no,,,"mask emission lines from lamp (affected pixels set to 1. in output)"
fl_inter,b,h,no,,,"Fit response interactively?"
fl_detec,b,h,no,,,"Fit response detector by detector rather than slit by slit?"
function,s,h,"spline3",spline3|legendre|chebyshev|spline1,,"Fitting function for response"
order,s,h,"15",,,"Order of fitting function, minimum value=1"
low_reject,r,h,3.,,,"Low rejection in sigma of response fit"
high_reject,r,h,3.,,,"High rejection in sigma of response fit"
niterate,i,h,2,,,"Number of rejection iterations in response fit"
combine,s,h,"average",,,"Combination operation"
reject,s,h,"avsigclip",,,"Rejection algorithm"
masktype,s,h,"goodvalue",none|goodvalue,,"Mask type"
maskvalue,r,h,0.,,,"Mask value"
scale,s,h,"mean",,,"Image scaling"
zero,s,h,"none",,,"Image zeropoint offset"
weight,s,h,"none",,,"Image weights"
statsec,s,h,"",,,"Statistics section"
lthreshold,r,h,INDEF,,,"Lower threshold"
hthreshold,r,h,INDEF,,,"Upper threshold"
nlow,i,h,1,0,,"minmax: Number of low pixels to reject"
nhigh,i,h,1,0,,"minmax: Number of high pixels to reject"
nkeep,i,h,0,0,,"Minimum to keep or maximum to reject"
mclip,b,h,yes,,,"Use median in sigma clipping algorithms?"
lsigma,r,h,3.,,,"Lower sigma clipping factor"
hsigma,r,h,3.,,,"Upper sigma clipping factor"
key_ron,s,h,"RDNOISE",,,"Keyword for readout noise in e-"
key_gain,s,h,"GAIN",,,"Keyword for gain in electrons/ADU"
ron,r,h,3.5,0.,,"Readout noise rms in electrons"
gain,r,h,2.2,1.0000000000000E-5,,"Gain in e-/ADU"
snoise,s,h,"0.0",,,"ccdclip: Sensitivity noise (electrons)"
sigscale,r,h,0.1,,,"Tolerance for sigma clipping scaling correction"
pclip,r,h,-0.5,,,"pclip: Percentile clipping parameter"
grow,r,h,0.,,,"Radius (pixels) for neighbor rejection"
ovs_flinter,b,h,no,,,"Interactive overscan fitting?"
ovs_med,b,h,no,,,"Use median instead of average in column bias?"
ovs_func,s,h,"chebyshev",spline3|legendre|chebyshev|spline1,,"Overscan fitting function"
ovs_order,i,h,1,,,"Order of overscan fitting function"
ovs_lowr,r,h,3.,,,"Low sigma rejection factor"
ovs_highr,r,h,3.,,,"High sigma rejection factor"
ovs_niter,i,h,2,,,"Number of rejection iterations"
logfile,s,h,"",,,"Logfile name"
verbose,b,h,yes,,,"Verbose"
status,i,h,1,,,"Exit status (0=good)"
scanfile,*struct,h,"",,,"Internal use only"
mode,s,h,"ql",,,
I call to gsflat doing epar gsflat, then I fill params to do finally :go.
I have write permission to write in the directory, I reduced the science frames in other directory, but i just made a mkiraf dir and none special procedure in both directories. The param list of gsflat is the following:
inflats,s,a,"gs1031075",,,"Input flatfields"
specflat,s,a,"fs1031075",,,"Output normalized flat (MEF)"
fl_answer,b,a,yes,,,"Continue interactive fitting?"
fl_slitcorr,b,h,no,,,"Correct output for Illumination/Slit-Function"
slitfunc,s,h,"",,,"Slit Function (MEF output of gsslitfunc)"
fl_keep,b,h,no,,,"Keep imcombined flat?"
combflat,s,h,"",,,"Filename for imcombined flat"
fl_over,b,h,yes,,,"Subtract overscan level"
fl_trim,b,h,yes,,,"Trim off overscan region"
fl_bias,b,h,yes,,,"Subtract bias image"
fl_dark,b,h,no,,,"Subtract (scaled) dark image"
fl_fixpix,b,h,no,,,"Interpolate across chip gaps"
fl_vardq,b,h,no,,,"Create variance and data quality frames"
bias,s,h,"gs1125_bias",,,"Bias image"
dark,s,h,"",,,"Dark image"
key_exptime,s,h,"EXPTIME",,,"Exposure time header keyword"
key_biassec,s,h,"BIASSEC",,,"Header keyword for overscan strip image section"
key_datasec,s,h,"DATASEC",,,"Header keyword for data section (excludes the overscan)"
rawpath,s,h,"",,,"GPREPARE: Path for input raw images"
sci_ext,s,h,"SCI",,,"Name of science extension"
var_ext,s,h,"VAR",,,"Name of variance extension"
dq_ext,s,h,"DQ",,,"Name of data quality extension"
key_mdf,s,h,"MASKNAME",,,"Header keyword for the MDF"
mdffile,s,h,"",,,"MDF to use if keyword not found"
mdfdir,s,h,"gmos$data/",,,"MDF database directory"
bpm,s,h,"",,,"Name of bad pixel mask file or image"
gaindb,s,h,"default",,,"Database with gain data"
gratingdb,s,h,"gmos$data/GMOSgratings.dat",,,"Gratings database file"
filterdb,s,h,"gmos$data/GMOSfilters.dat",,,"Filters database file"
bpmfile,s,h,"gmos$data/chipgaps.dat",,,"Info on location of chip gaps"
refimage,s,h,"",,,"Reference image for slit positions"
sat,r,h,65000.,,,"Saturation level in raw images"
xoffset,r,h,INDEF,,,"X offset in wavelength [nm]"
yoffset,r,h,INDEF,,,"Y offset in unbinned pixels"
yadd,r,h,0.,,,"Additional pixels to add to each end of MOS slitlet lengths"
fl_usegrad,b,h,no,,,"Use gradient method to find MOS slits"
fl_emis,b,h,no,,,"mask emission lines from lamp (affected pixels set to 1. in output)"
fl_inter,b,h,no,,,"Fit response interactively?"
fl_detec,b,h,no,,,"Fit response detector by detector rather than slit by slit?"
function,s,h,"spline3",spline3|legendre|chebyshev|spline1,,"Fitting function for response"
order,s,h,"15",,,"Order of fitting function, minimum value=1"
low_reject,r,h,3.,,,"Low rejection in sigma of response fit"
high_reject,r,h,3.,,,"High rejection in sigma of response fit"
niterate,i,h,2,,,"Number of rejection iterations in response fit"
combine,s,h,"average",,,"Combination operation"
reject,s,h,"avsigclip",,,"Rejection algorithm"
masktype,s,h,"goodvalue",none|goodvalue,,"Mask type"
maskvalue,r,h,0.,,,"Mask value"
scale,s,h,"mean",,,"Image scaling"
zero,s,h,"none",,,"Image zeropoint offset"
weight,s,h,"none",,,"Image weights"
statsec,s,h,"",,,"Statistics section"
lthreshold,r,h,INDEF,,,"Lower threshold"
hthreshold,r,h,INDEF,,,"Upper threshold"
nlow,i,h,1,0,,"minmax: Number of low pixels to reject"
nhigh,i,h,1,0,,"minmax: Number of high pixels to reject"
nkeep,i,h,0,0,,"Minimum to keep or maximum to reject"
mclip,b,h,yes,,,"Use median in sigma clipping algorithms?"
lsigma,r,h,3.,,,"Lower sigma clipping factor"
hsigma,r,h,3.,,,"Upper sigma clipping factor"
key_ron,s,h,"RDNOISE",,,"Keyword for readout noise in e-"
key_gain,s,h,"GAIN",,,"Keyword for gain in electrons/ADU"
ron,r,h,3.5,0.,,"Readout noise rms in electrons"
gain,r,h,2.2,1.0000000000000E-5,,"Gain in e-/ADU"
snoise,s,h,"0.0",,,"ccdclip: Sensitivity noise (electrons)"
sigscale,r,h,0.1,,,"Tolerance for sigma clipping scaling correction"
pclip,r,h,-0.5,,,"pclip: Percentile clipping parameter"
grow,r,h,0.,,,"Radius (pixels) for neighbor rejection"
ovs_flinter,b,h,no,,,"Interactive overscan fitting?"
ovs_med,b,h,no,,,"Use median instead of average in column bias?"
ovs_func,s,h,"chebyshev",spline3|legendre|chebyshev|spline1,,"Overscan fitting function"
ovs_order,i,h,1,,,"Order of overscan fitting function"
ovs_lowr,r,h,3.,,,"Low sigma rejection factor"
ovs_highr,r,h,3.,,,"High sigma rejection factor"
ovs_niter,i,h,2,,,"Number of rejection iterations"
logfile,s,h,"",,,"Logfile name"
verbose,b,h,yes,,,"Verbose"
status,i,h,1,,,"Exit status (0=good)"
scanfile,*struct,h,"",,,"Internal use only"
mode,s,h,"ql",,,
I call to gsflat doing epar gsflat, then I fill params to do finally :go.
emma wrote on Jun 10, 2010
Hello again Jorge :)
From your parameter list, it looks like you are not using the latest version of the Gemini data reduction package ... can you confirm what version you are using? The latest version (v1.10) can be found here: http://www.gemini.edu/sciops/data-and-results/processing-software/. If you are not using the latest version I would recommend upgrading, since the latest version contains many improvements to the tasks in the GMOS package.
If you are already using the latest version, please make sure you "unlearn gemini" before running any tasks ... this will remove any old Gemini parameter files that are no longer applicable to the current tasks.
If you are still having problems, please let me know. Just one last question, for clarification ... did you run "mkiraf" once (and once only) to create a single login.cl and uparm directory? Or did you run mkiraf more than once i.e., are you using one login.cl and uparm directory for your science data reduction and a separate, different login.cl and uparm directory for your standard star data reduction?
Many thanks,
Emma :)
From your parameter list, it looks like you are not using the latest version of the Gemini data reduction package ... can you confirm what version you are using? The latest version (v1.10) can be found here: http://www.gemini.edu/sciops/data-and-results/processing-software/. If you are not using the latest version I would recommend upgrading, since the latest version contains many improvements to the tasks in the GMOS package.
If you are already using the latest version, please make sure you "unlearn gemini" before running any tasks ... this will remove any old Gemini parameter files that are no longer applicable to the current tasks.
If you are still having problems, please let me know. Just one last question, for clarification ... did you run "mkiraf" once (and once only) to create a single login.cl and uparm directory? Or did you run mkiraf more than once i.e., are you using one login.cl and uparm directory for your science data reduction and a separate, different login.cl and uparm directory for your standard star data reduction?
Many thanks,
Emma :)
jorge diaz wrote on Jun 10, 2010
I have updated mi version of gemini, and I still have the same problem. I don't know how to unlearn so I have deleted the old gemini files and put the new files in the extern directory. Then, i deleted the param files from uparm directory and call to iraf without problem. I call gemini, gmos package and then edit gsflat param and run, but the problems still continue.
I made a new mkiraf for the directory where i put the standard star files.
Do you have free time for check my images. I can upload some images for you can check them and see if the problem is iraf or the images.
Thank you and regards.
jorge.
I made a new mkiraf for the directory where i put the standard star files.
Do you have free time for check my images. I can upload some images for you can check them and see if the problem is iraf or the images.
Thank you and regards.
jorge.
emma wrote on Jun 10, 2010
Hi Jorge,
Sorry for my delayed reply, I have been travelling and have just arrived back in the office today.
When I said to "unlearn gemini", I meant to remove the parameter files in the uparm directory, which you already did :) All unlearn does is remove those files from within IRAF, so you don't need to manually delete the files in the uparm directory.
Yes, I can take a look at your images, no problem :) All you need to do is tell me the file names of the images you want me to look at. If you do not want to post the image names on the forum, feel free to send me a private message. Please can you also post the commands you have run on the raw data so that I can try and duplicate your problem.
Many thanks,
Emma :)
Sorry for my delayed reply, I have been travelling and have just arrived back in the office today.
When I said to "unlearn gemini", I meant to remove the parameter files in the uparm directory, which you already did :) All unlearn does is remove those files from within IRAF, so you don't need to manually delete the files in the uparm directory.
Yes, I can take a look at your images, no problem :) All you need to do is tell me the file names of the images you want me to look at. If you do not want to post the image names on the forum, feel free to send me a private message. Please can you also post the commands you have run on the raw data so that I can try and duplicate your problem.
Many thanks,
Emma :)
jorge diaz wrote on Jun 10, 2010
Hi emma, have you had time to see the frames... have you found the problem?.
Best.
jorge.
Best.
jorge.
emma wrote on Jun 10, 2010
Hi Jorge,
Sorry again for my delayed reply, I was travelling again ;)
I started a new iraf session:
loaded the gemini and gmos packages:
initialised my uparm directory (I ran this command from the directory where my uparm directory is located):
then checked that it was empty:
the output of which was:
then ran the following command:
I actually downloaded your raw data from the archive and ran gsflat on the raw data (in place of <yourfile75> above), rather than the data you sent me. I did, however, use the bias frame you sent me (in place of <yourbias> above).
gsflat ran without any errors. However, I did get a:
message though, so I don't think the bias frame you sent me is correct for the standard star data you sent me.
The problem might be that you are not running gsflat on the raw data. Other than that, I think it must be something to do with your installation, since I can reduce the data just fine here. You should only need to run mkiraf once so that you have a single login.cl file and a single uparm directory. When you start IRAF, check that everything is set correctly by typing the following:
Make sure that the output for this is the path to where your login.cl file is located. Then try:
This should return:
and means that the uparm directory is defined to be in the directory where your login.cl is located.
Also just double check you have write permissions for your uparm directory and also in the directory you are working in.
Emma :)
Sorry again for my delayed reply, I was travelling again ;)
I started a new iraf session:
clloaded the gemini and gmos packages:
gemini
gmosinitialised my uparm directory (I ran this command from the directory where my uparm directory is located):
rm -rf uparm/*then checked that it was empty:
dir uparm$the output of which was:
no files foundthen ran the following command:
gsflat inflats=<yourfile75>.fits specflat=f<yourfile75>.fits fl_over+ fl_fixpix- bias=<yourbias>.fitsI actually downloaded your raw data from the archive and ran gsflat on the raw data (in place of <yourfile75> above), rather than the data you sent me. I did, however, use the bias frame you sent me (in place of <yourbias> above).
gsflat ran without any errors. However, I did get a:
WARNING - GIREDUCE: Mismatch detector section or binning of biasmessage though, so I don't think the bias frame you sent me is correct for the standard star data you sent me.
The problem might be that you are not running gsflat on the raw data. Other than that, I think it must be something to do with your installation, since I can reduce the data just fine here. You should only need to run mkiraf once so that you have a single login.cl file and a single uparm directory. When you start IRAF, check that everything is set correctly by typing the following:
show homeMake sure that the output for this is the path to where your login.cl file is located. Then try:
show uparmThis should return:
home$uparmand means that the uparm directory is defined to be in the directory where your login.cl is located.
Also just double check you have write permissions for your uparm directory and also in the directory you are working in.
Emma :)
Last post on Jun 10, 2010