New User, New Programmer, Help?

bearsindc wrote on Aug 14, 2009

I'm a University student and I'm very green when it comes to programming and scripting. I'm attempting to develop a script that will automatically generate median pixel images from a group of dark field images. I've developed my script so far that it deposits the appropriate images into list sets (which I can then manually turn into median images using imcombine @list ... etc.). What I'm having trouble with is determining the imcombine syntax in the CL script, and I would like to place all of my lists into a 'master list' to be read into the imcombine task. Here's my script so far, with the last few lines commented out:

string *list1
string *list2
string *list3
string *list4

real temp
real exp_array[25] = 25(0)
int darkexp_ctr = 1
int ctr = 1
real exp

list1 = "RawDark.lis"
list2 = "DarkMEDTIME.txt"
list3 = "DarkMedMaster.lis"
list4 = "DarkMEDTIME.txt"

while(fscan(list1 , s1) != EOF)
{
imgets(s1 , "EXPTIME")
if (ctr == 1) {
print(s1 , " " , imgets.value, > list2)
ctr=ctr+1
}
else {
print(s1 , " " , imgets.value, >> list2)
}
}
while(fscan(list4 , s1 , s2) != EOF)
{
exp = real(s2)
if (exp != exp_array[1] && exp != exp_array[2] && exp != exp_array[3] && exp != exp_array[4] && exp != exp_array[5] && exp != exp_array[6] && exp != exp_array[7] && exp != exp_array[8] && exp != exp_array[9] && exp != exp_array[10] && exp != exp_array[11] && exp != exp_array[12] && exp != exp_array[13] && exp != exp_array[14] && exp != exp_array[15] && exp != exp_array[16] && exp != exp_array[17] && exp != exp_array[18] && exp != exp_array[19] && exp != exp_array[20] && exp != exp_array[21] && exp != exp_array[22] && exp != exp_array[23] && exp != exp_array[24] && exp != exp_array[25])
{
print(s1, > s2//'list')
exp_array[darkexp_ctr]=exp
darkexp_ctr=darkexp_ctr+1
}
else {
print(s1, >> s2//'list')
}
}

#pipe all output to single 'list3'
#possible to wrap entire statement in a 'while' loop?
#use while loop to dump all output?

#while(fscan(list3, s1) != EOF)
#{
#imcombine(@ "s1", combine = median, reject = none)
#}
#imcombine([@listin] [fileout] combine = [combtype])
#what is i/o name? what is name of output? does input require '@'?

This may be an elementary question to most, but I'm running out of ideas. Thanks!

---

Mike Fitzpatrick wrote on Aug 14, 2009

If you want to pass an @file into the task that uses a script variable name, then you need to use string concatenation to form the parameter, e.g.

imcombine ("@"//s1, ......)

None also that when using the "program mode" syntax like this, the string params need to be quoted so they won't be confused with variables, e.g.

imcombine ("@"//s1, combine="median", reject="none", ,,,,,,)

-Mike

---

bearsindc wrote on Aug 14, 2009

Thanks for the help Mike!

I'm still having trouble developing a way to read the lists that i generated in the first portion of the script.

At the moment the script generates a number of lists corresponding to the number of unique exposure times available in the raw images. Each list file contains 5-6 images. Now that I've generated the lists I need the script to insert each one of these lists separately into the imcombine input parameter and generate the median of those images. My first thought on how to go about this was to create a "list of lists." This master list can be scanned and each list itemized as a string variable (please don't take offense if I misuse jargon, it would be really helpful to learn some of it) to be used by imcombine.

My first attempt wrote to the master only when the first list was created. For each successive list the data was appended. However, I received errors stating that some files could not be opened properly.

Is there an easier way to make all of my output go to a single list? Is this "list of lists" the appropriate route?

Thanks!

---

Mike Fitzpatrick wrote on Aug 14, 2009

A more general approach might be to 1) create a list of unique exposure times, and 2) create lists of images that have each of those exposure times.
You can do this using the HSELECT task in the following ways, e.g.

To create a list of unique exposure times:

hselect ("*.fits", "EXPTIME", yes) | sort ("STDIN") | uniq ("STDIN", > "exptimes.txt")

The file 'exptimes.txt' then contains a list of all the unique exposure times for all the files in the directory. So, if you have a 100 files but the only EXPTIMES are 300 or 600 seconds, this file will have just those two values.

The next step is to find those images that have each exposure time, e.g.

list = "exptimes.txt"
i = 1
while (fscan (list, tim) != EOF) {
    hselect ("*.fits", "$I", "exptime?='" // tim // "'", > "flist"//i)
    i = i + 1
}

This scans the file and for each exposure time calls HSELECT to print out the image name if the EXPTIME matches the value read. This may look a little advanced (e.g. the '?=' string comparator, see the HSELECT and HEDIT help pages), but you'll end up with a series of files starting with the string 'flist' that each have the same exposure time. From there you could likewise loop over each of these files to call IMCOMBINE on that list.

Hope this gives you an option.
-Mike

---

bearsindc wrote on Aug 14, 2009

Again, you've made my life that much easier. This stuff is coming a bit more naturally now.

---

bearsindc wrote on Aug 14, 2009

ERROR: operands have incompatible types
"hselect("*.fits" , "$I" , "EXPOSURE?='" // s1 // "'", > "flist"//i)"

i've been playing around with your suggestions and i've run into this error. i'm assuming it's because the variable s1 is read by hselect as something other than a string (this i'm guessing because the ?= is a string comparator)?

here's the entire code:

string *list1
string *list2

real temp

list1 = "RawDark.lis"
list2 = "ExpTime.txt"

print ("yo")

while(fscan("list1" , s1) != EOF)
{
print (s1)
print ("hello")
hselect("*.fits" , "EXPOSURE" , yes) | sort ("STDIN") | uniq ("STDIN" , > "ExpTime.txt")
print ("goodbye")
}
print ("hi")
i = 1
while(fscan(list2 , s1) != EOF)
{
print (s1)
hselect("*.fits" , "$I" , "EXPOSURE?='" // s1 // "'", > "flist"//i)
i = i + 1
}

excuse the miscellaneous print statements, they're my debugging tools.

---

Mike Fitzpatrick wrote on Aug 14, 2009

The error may be because the EXPOSURE keyword value is an int/float in the image header and not a quoted string. If so, the '==' operator can be used instead of '?='

-Mike

---

bearsindc wrote on Aug 14, 2009

The error is the same even with the '==' comparator. Could the problem be the declaration 'real temp?'

---

Mike Fitzpatrick wrote on Aug 14, 2009

Sorry, I was being hasty. Regardless of the operator you use, the right side of the expression is still using a quoted value and is a string, so if the EXPOSURE value is an int you're comparing with a string and thus the message. Try removing the quotes, or post an image header and I can be more specific.

---

bearsindc wrote on Aug 14, 2009

actually, i was able to get the script to run by cutting out a few of the quotations and backslashes.

the working syntax for the hselect command is:

i = 1
print("hi")
print("hello")
hselect("Dark*.fits" , "$I" , "EXPOSURE=="//s1, > "flist"//i)
i = i + 1

disregarding the print statements again.

thanks!

---

Last post on Aug 14, 2009