IUE data reduction in IRAF
Frank Valdes wrote on Jun 08, 1998
Hi Petr, Let me start by saying that combining echelle orders well is a difficult problem and is not well addressed by the IRAF/NOAO echelle software. The problem you reported about the physical basis for nonlinear dispersion functions arises because of the way the "multispec" format stores multiple spectra in an image with a single header. While it can handle independent nonlinear dispersion functions they must all be tied to the same pixel system. Without attempting to explain in detail, when you cut different orders at different points you encounter this limitation. You approach of copying to separate files is a reasonable, though tedious, solution. I understand your desire to maintain the original "non-linear" dispersions but if you need to do things with spectral "images" and trying to combine them into a single spectral image then you really have no choice but to linearize. (My own personal opinion is also that this generally has no scientific impact on the data and people become overly fixated on this.) So you might as well do this at an early stage. Rather than using SCOPY to "cut" the orders I would suggest you take the single image echelle format multispec image and use DISPCOR to resample to a uniform linear sampling for all orders. I understand that this means either some orders are oversampled or some are undersampled but you can choose. This will have several benefits including addressing how to cut orders so that two pixels don't overlap. Since all pixels have the same dlambda relation this should be straightforward now. As you noted SCOMBINE, and most current IRAF/NOAO tasks don't do much with bad pixel masks. This is unfortunate and we recognize this lack in the software and hope to make progress on it. How do you want to use the quality flags in this application? There are probably other ways to accomplish what you want to do. I cannot address alternate ways to deal with spectra as tables. I would think that it should be possible to simply make a new table by merging the orders as a simple list of wavelength+flux but I don't know. If you ever get the data in that single table then converting to a raster spectrum, with/without resampling, should be possible. As you may realize the NOAO spectral software was developed around the model of a spectrum as a raster image while other groups, such as STSDAS, are implementing a table model. The table model may have some advantages but, as you are finding, the body of analysis and data handling software is much more limited. I don't think there is a significant readership of this newsgroup by those who support the table spectral model but this is recognized as a problem by the NOAO IRAF User's Committee. You should certainly pursue this with the suppliers of the data format through direct e-mail so they know what problems the user's of the format encounter. There may be software around that would be useful but I am not aware of it. I support (part-time) the NOAO spectral software (including echelle) so that is what I can tell you about in detail. It would be nice if the NOAO spectral software would evolve to handle spectra as tables but since there is currently much less than one person of manpower for this software it is not likely to change anytime soon. If you do develop any generally useful IRAF code I'm sure it would be of value to others but this is not a simple thing to do for compiled or fundamental code without some close support from current software developers. However, scripts based on the existing tables and spectral analysis tools are certainly a doable and useful thing to share. I would be glad to answer questions and help you in some limited way. It would be nice if this newsgroup could be used to have a good exchange of knowledge on echelle issues, IUE data handling, and spectral software but currently this is not likely and you will need to write to iraf@noao.edu for the NOAO software (though I certainly read the newsgroup) and HST or others for the IUE tables data handling issues. Posting here or copying me would help me to increase my knowledge of these issues. Cheers, Frank Valdes NOAO/IRAF Group > From adass-iraf-applications@tucana Tue May 26 10:24:49 1998 > Date: Tue, 26 May 98 10:24:48 MST > Reply-To: adass-iraf-applications@tucana > Originator: adass-iraf-applications@iraf.noao.edu > From: irafnet > To: valdes > Subject: IUE data reduction in IRAF > > Hello everybody, > > I am trying to work with the IUE high dispersion data (MXHI) in IRAF > and have a lot of questions for IRAF gurus: > > The data are in binary table, which is then translated by the iuetools > package into binary table FITS with lambda expanded array then OIF > multispec format. There are 60 orders of echelle data in it - each > order in separate line . The header contains the dispersion solution for > each order. Now problems: > > As the data from neighbouring orders do overlap, there must be choosen > limits for each order and the order data cut at these borders. For > this I use the algorithms by Solano (1998) > which will take the one third of the overlaping region from order m and > the rest from m-1. > >From these data I want to have one large spectrum with concatenated > orders. This task doesn't solve the iuetools package. > > I could find only task scopy for extracting cuted orders and scombine > for glueing together the orders. However there are problems : > > Here is a part of code (with the evaluated value of variables for the > clarity) > > > > scopy (input=swp47913.imh, output=sum47913, w1=1173.8600132649, > w2=1183.8862202, apertures=, beams=117, apmodulus=0, format=multispec, > renumber=no, offset=0, clobber=yes, merge=yes, rebin=no, verbose=yes) > > and after one run of cycle > > scopy (input=swp47913.imh, output=sum47913, w1= 1183.8862202, > w2=1194.08880079, apertures=, beams=116, apmodulus=0, format=multispec, > renumber=no, offset=0, clobber=yes, merge=yes, rebin=no, verbose=yes) > > This doesn't work - there is a complain "Warning: Physical basis for > nonlinear dispersion functions don't match" and then WFMSPEC: No > dispersion function; > > I understand that the problem is in different dispersion relation, but I > can't switch rebin=yes, as this leads to the error segmentation > violation - recently announced bug. > > I am afraid there is no solution for having the scombine produce the > multispec image with disjunctly cutted orders (and scombine them in the > image), so I switched format=onedspec > and then scombine each individual file with orders and combine=average. > > There is a another problem with the combining . When I cut the regions > as shown above > at the same value I have two data points in that particular wavelength - > when I sum them a peak is produced, when average and intermediate point > is there. I would like to have the cutted spectra realy disjunct (so > that even the combine=sum will not count two values) - so I need to fit > the edges to some lambda_cut-dlambda, to avoid the last pixel in > data, but I do not know how to set the dlambda to represent only the > last pixel - value in data. > I can set some adhoc value, but if it is to large it could ignore more > pixels - and it will not probably be the same for all orders. > > I suppose that such a problem of concatenting overlaping echelle oreders > must be tackled in IRAF , however I could not find anything in > description of echelle package nor in the echelle cookbook (reducing > echelle spectra with IRAF or so - I do not remember exactly). > > Can you recommend me a solution ? > > The last problem - the IUE data include the quality flags when the > unreliable value of flux is marked. I would like to produce from it a > kind of badpixel map, but I couldnt find how to apply it to scombine (in > imcombine I have the fix+,but I do not work with pixels but wavelengths > in scopy ) > > I also would like to know an alternative solution of working with the > original IUE FITS - there are binary tables (after conversion of > mxexpand routine ) with columns of wavelength values, flux, noise , > quality flags etc. Is it possible to catenate the spectra at this > level without rebining to wavelength, correctly mask out the bad pixels > according to quality flags and then convert to one long OIF spectrum to > be able to use rv package for measuring radial velocity ? Is there even > possible to measure RV and identify lines in STSDAS using the binary > tables directly ? > > I am just getting acquitant with echelle data and IUE specifics and have > read a lot of IRAF and STSDAS helps but the right solution still avoids > me ;-) . > > I belive in a power of IRAF to be able to handle all these problems ( > I might try to write some package code if you will show me the possible > way) > > Regards Petr Skoda > > > >
Last post on Jun 08, 1998