ccdproc file list limitations

parejkoj wrote on Apr 09, 2009

Another question regarding running things via pyraf:

When I pass a long file list to ccdproc from my python code, like so:

    biases = ['../'+base+'%04i.fits'%x for x in range(b0,b1+1)]
    files = glob.glob('../'+base+'*[0-9].fits')
    files = [x for x in files if x not in biases]
    outfiles = [x[3:] for x in files] # strips off the '../'

    ccdred.ccdproc(','.join(files[:n]),output=','.join(outfiles[:n]),
		   overscan=True,trim=True,zerocor=True,readaxis='column',
		   biassec=biassec,trimsec=trimsec,interactive='no',
		   zero=biasname())

if files (and thus outfiles) contains more than 51 elements, ccdproc bails with the following error:

Input and output lists do not match

Is this a known limitation in the length of parameter lists, or is this a bug? And if it is a limitation, where is documented as such? And why can't I send a longer parameter list? When I turn verbosity on, the epars look correct:

found ccdred.ccdproc in task list
run ccdproc (IrafTask: /usr/stsci/iraf/iraf/noao/bin.macintel/x_ccdred.e)
       images = "../Feb13/b0001.fits,../Feb13/b0002.fits,../Feb13/b0018.fits,../Feb13/b0019.fits,
../Feb13/b0020.fits,../Feb13/b0021.fits,../Feb13/b0022.fits,../Feb13/b0023.fits,
../Feb13/b0024.fits,../Feb13/b0025.fits,../Feb13/b0026.fits,../Feb13/b0027.fits,
../Feb13/b0028.fits,../Feb13/b0029.fits,../Feb13/b0030.fits,../Feb13/b0031.fits,
../Feb13/b0032.fits,../Feb13/b0033.fits,../Feb13/b0034.fits,../Feb13/b0035.fits,
../Feb13/b0036.fits,../Feb13/b0037.fits,../Feb13/b0038.fits,../Feb13/b0039.fits,
../Feb13/b0040.fits,../Feb13/b0041.fits,../Feb13/b0042.fits,../Feb13/b0043.fits,
../Feb13/b0044.fits,../Feb13/b0045.fits,../Feb13/b0046.fits,../Feb13/b0047.fits,
../Feb13/b0048.fits,../Feb13/b0049.fits,../Feb13/b0050.fits,../Feb13/b0051.fits,
../Feb13/b0052.fits,../Feb13/b0053.fits,../Feb13/b0054.fits,../Feb13/b0055.fits,
../Feb13/b0056.fits,../Feb13/b0057.fits,../Feb13/b0058.fits,../Feb13/b0059.fits,
../Feb13/b0060.fits,../Feb13/b0061.fits,../Feb13/b0062.fits,../Feb13/b0063.fits,
../Feb13/b0064.fits,../Feb13/b0065.fits,../Feb13/b0066.fits,../Feb13/b0067.fits" List of CCD images to correct
      (output = "Feb13/b0001.fits,Feb13/b0002.fits,Feb13/b0018.fits,Feb13/b0019.fits,Feb13/b0020.fits,
Feb13/b0021.fits,Feb13/b0022.fits,Feb13/b0023.fits,Feb13/b0024.fits,Feb13/b0025.fits,
Feb13/b0026.fits,Feb13/b0027.fits,Feb13/b0028.fits,Feb13/b0029.fits,Feb13/b0030.fits,
Feb13/b0031.fits,Feb13/b0032.fits,Feb13/b0033.fits,Feb13/b0034.fits,Feb13/b0035.fits,
Feb13/b0036.fits,Feb13/b0037.fits,Feb13/b0038.fits,Feb13/b0039.fits,Feb13/b0040.fits,
Feb13/b0041.fits,Feb13/b0042.fits,Feb13/b0043.fits,Feb13/b0044.fits,Feb13/b0045.fits,
Feb13/b0046.fits,Feb13/b0047.fits,Feb13/b0048.fits,Feb13/b0049.fits,Feb13/b0050.fits,
Feb13/b0051.fits,Feb13/b0052.fits,Feb13/b0053.fits,Feb13/b0054.fits,Feb13/b0055.fits,
Feb13/b0056.fits,Feb13/b0057.fits,Feb13/b0058.fits,Feb13/b0059.fits,Feb13/b0060.fits,
Feb13/b0061.fits,Feb13/b0062.fits,Feb13/b0063.fits,Feb13/b0064.fits,Feb13/b0065.fits,
Feb13/b0066.fits,Feb13/b0067.fits") List of output CCD images
     (ccdtype = "")             CCD image type to correct

Thank you

---

Mike Fitzpatrick wrote on Apr 09, 2009

There is no limit on the number of files that CCDPROC can process, however your input image list DOES exceed the maximum length of a CL parameter string (1024) and this is probably the cause. To fix it, write the filenames to some file and pass it in as an @list of the filenames to process, or else use some sort of template like "file*.fits".

-Mike

---

parejkoj wrote on Apr 09, 2009

Are there plans to allow for longer CL parameter strings? It would be very handy to be able to pass arbitrary parameter lists in that manner.

---

Mike Fitzpatrick wrote on Apr 09, 2009

Arbitrarily long string parmeters can't be done unless every application that has a string parameter is changed as well. If your parameter exceeds the length then you should either be using an @file or a template that can be arbitrarily expanded (e.g. "file*.fits").

---

Last post on Apr 09, 2009