DISPCOR and nfit1d
Sparkles wrote on Oct 08, 2010
Hi,
I'm attempting to bin my data, calculate the standard deviations and see how well it fits. I've been on it for a while and a bit stuck. My plan so far:
Binned data is fine
step 2 - from the mean values of the 16 points I need to calculate the standard deviation for the flux, i.e the standard deviation with respect to the continuum points (that lie in the 50 angstrom bins) and the mean value (calculated earlier in step one).
does anyone know of a package that can do this?? I've been looking into imstat - but not much luck I'm afraid. I don't know how to tell Iraf take the Flux values in this range (50A) and the mean Flux value XXX and calculate the standard deviation.
Step 3 - Is there also a way of feeding in the error (step 2) to get a better result for chisq?
Sorry to keep bothering up all I'm sure you have lots to do. I've been using Iraf on and off for two weeks now and its a very steep learning curve! And I'm trying to piece things together through threads the help menu and research papers :?
Thanks S
I'm attempting to bin my data, calculate the standard deviations and see how well it fits. I've been on it for a while and a bit stuck. My plan so far:
Binned data is fine
step 2 - from the mean values of the 16 points I need to calculate the standard deviation for the flux, i.e the standard deviation with respect to the continuum points (that lie in the 50 angstrom bins) and the mean value (calculated earlier in step one).
does anyone know of a package that can do this?? I've been looking into imstat - but not much luck I'm afraid. I don't know how to tell Iraf take the Flux values in this range (50A) and the mean Flux value XXX and calculate the standard deviation.
Step 3 - Is there also a way of feeding in the error (step 2) to get a better result for chisq?
Sorry to keep bothering up all I'm sure you have lots to do. I've been using Iraf on and off for two weeks now and its a very steep learning curve! And I'm trying to piece things together through threads the help menu and research papers :?
Thanks S
Francisco Valdes wrote on Oct 08, 2010
Hi,
Indeed IRAF provides such a large collection of tools that it is not something that you can master in a few weeks. However, it seems you've done pretty well already.
I think I understand what you want to do and it is an interesting question. Ideally the same task, blkavg, should have an option to output a standard deviation instead of or in addition to the average. This would be an really good addition. However, that is not currently an option.
So we have to be a little more clever. There is a general task, raverage, that provides a running average with standard deviation. I realize you are not asking for a running value but discrete bins. This just means you would have to take every Nth value (where N is the window) to get the independent samples. Raverage works off of a text file so you would first need to convert the spectrum to a text file. Note that I am assuming you have a simple 1D spectral image.
From the out file you could then convert pairs of columns back to 1D images or use the graph task with overlays or write your own script. I would call your attention to the fields task as a very useful tool to manipulate text files. The TTOOLS package in TABLES or STSDAS package can do a lot of things but it is a more complicated "database" like tools.
As for the chi2 stuff, maybe you really would benefit from the TTOOLS package or else you should learn how to program the CL.
I hope this helps.
Frank Valdes
Indeed IRAF provides such a large collection of tools that it is not something that you can master in a few weeks. However, it seems you've done pretty well already.
I think I understand what you want to do and it is an interesting question. Ideally the same task, blkavg, should have an option to output a standard deviation instead of or in addition to the average. This would be an really good addition. However, that is not currently an option.
So we have to be a little more clever. There is a general task, raverage, that provides a running average with standard deviation. I realize you are not asking for a running value but discrete bins. This just means you would have to take every Nth value (where N is the window) to get the independent samples. Raverage works off of a text file so you would first need to convert the spectrum to a text file. Note that I am assuming you have a simple 1D spectral image.
cl> wspectext myspec myspec.txt header-
cl> raverage myspec.txt 16 > out.txt
From the out file you could then convert pairs of columns back to 1D images or use the graph task with overlays or write your own script. I would call your attention to the fields task as a very useful tool to manipulate text files. The TTOOLS package in TABLES or STSDAS package can do a lot of things but it is a more complicated "database" like tools.
As for the chi2 stuff, maybe you really would benefit from the TTOOLS package or else you should learn how to program the CL.
I hope this helps.
Frank Valdes
Sparkles wrote on Oct 08, 2010
Thanks Frank, That worked out great
I have also found that nfit1d (stsdas) is able to fit a power law function and find the corresponding chi squared value. However the errors (standard deviations, sd) must first be entered, else a default of 1 is used for all sd.
In order to do this the erropar setting (epar nfit1d) needs a table input.However when I give it the table ( used convert to convert from txt to tab file) and try nfit1d it just states:
But its in the same directory - so I'm really confused!! I don't know if you know much about the stsdas stuff??
Thanks, S
I have also found that nfit1d (stsdas) is able to fit a power law function and find the corresponding chi squared value. However the errors (standard deviations, sd) must first be entered, else a default of 1 is used for all sd.
In order to do this the erropar setting (epar nfit1d) needs a table input.However when I give it the table ( used convert to convert from txt to tab file) and try nfit1d it just states:
error: task "tablename" not foundBut its in the same directory - so I'm really confused!! I don't know if you know much about the stsdas stuff??
Thanks, S
Francisco Valdes wrote on Oct 08, 2010
Hi,
The "errorpars" parameter in NFIT1D is a more sophisticated type of parameter than you are aware of. In the help it notes this is a "pset". This means that the value points to another parameter set. Normally you leave this blank and it uses the parameters for a task of the same name, i.e. errorpars. So in this case you need to do "epar errorpars" (and read "help errorpars") and set those parameters.
Note that one feature of a pset parameter is that you can put the errorpars parameters directly on the command line (without the errorpars. prefix) such as
The errorpars.errcolumn is where you specify the column for the errors. This can be complicated but you are making good progress.
Frank
The "errorpars" parameter in NFIT1D is a more sophisticated type of parameter than you are aware of. In the help it notes this is a "pset". This means that the value points to another parameter set. Normally you leave this blank and it uses the parameters for a task of the same name, i.e. errorpars. So in this case you need to do "epar errorpars" (and read "help errorpars") and set those parameters.
Note that one feature of a pset parameter is that you can put the errorpars parameters directly on the command line (without the errorpars. prefix) such as
nfit1d ... errcolumn="..."
The errorpars.errcolumn is where you specify the column for the errors. This can be complicated but you are making good progress.
Frank
Sparkles wrote on Oct 08, 2010
Hi Frank,
I see what you mean. I need to go into epar errorpar and set the errcolumn. How does Iraf know which table to look into?? I've converted the txt files to tab and the arrangement is as follows:
wavelength Flux Error
However setting errorcolumn = error and made sure that this is the only table in the directory.
My supervisor now wants me to write an entire python script by the end of the day to fit the power law and calculate chi-squared (among my other 101 million jobs - taking a masters was a bad choice :wink: ), but I don't see the point in wasting time if I can get Iraf can do it??
Any Ideas welcome?? Thanks again.
I see what you mean. I need to go into epar errorpar and set the errcolumn. How does Iraf know which table to look into?? I've converted the txt files to tab and the arrangement is as follows:
wavelength Flux Error
However setting errorcolumn = error and made sure that this is the only table in the directory.
My supervisor now wants me to write an entire python script by the end of the day to fit the power law and calculate chi-squared (among my other 101 million jobs - taking a masters was a bad choice :wink: ), but I don't see the point in wasting time if I can get Iraf can do it??
Any Ideas welcome?? Thanks again.
Francisco Valdes wrote on Oct 08, 2010
I'm sorry you are getting so much pressure.
The simplest thing is to make a text file with three columns, X, Y, and error. You then don't have to set anything in error pars since nfit1d will know that the errors are the third column. So don't bother with the table format and don't bother with the errorpars. However, the package and tools are quite flexible and you can do things with tables and named columns, etc. I think if you have a table, but I haven't tried this, you only specify the name of the error column in errorpars and it is assumed to be in the input table specified to nfit1d.
My test was to make a text file with x=1-100, y=random, err=0.01 and fit a user function of a line. I got a chisq of 690. I then set the errors to 0.1 and got a chisq of 6. So I'm pretty sure it is reading the error column as I described.
To look at the output results I was using tprint (the print task in the tables package).
I hope this helps.
Frank
The simplest thing is to make a text file with three columns, X, Y, and error. You then don't have to set anything in error pars since nfit1d will know that the errors are the third column. So don't bother with the table format and don't bother with the errorpars. However, the package and tools are quite flexible and you can do things with tables and named columns, etc. I think if you have a table, but I haven't tried this, you only specify the name of the error column in errorpars and it is assumed to be in the input table specified to nfit1d.
My test was to make a text file with x=1-100, y=random, err=0.01 and fit a user function of a line. I got a chisq of 690. I then set the errors to 0.1 and got a chisq of 6. So I'm pretty sure it is reading the error column as I described.
To look at the output results I was using tprint (the print task in the tables package).
I hope this helps.
Frank
Last post on Oct 08, 2010