IRAF 2.16 mscred ccdproc problem

Robert Janusz wrote on Mar 24, 2012

I successfully installed IRAF 2.16 and got problems with the mscred package.
I can run:

mscred> ccdlist b0212

and I get:

b0212[im1][1036,2032][ushort][][1][]:SYM6712
b0212[im2][1036,2032][ushort][][2][]:SYM6712

with successful "setinstr" (as in v. 2.15, it worked fine), I run:

ccdproc b0212 overs+ trim+ merge-

and I get:

ERROR: Cannot open file (b0212)
  "imcopy (input//"[0]", outname, verbose=no)"
     line 288: mscsrc$ccdproc.cl
     called as: `ccdproc (images=b0212, overscan=yes, trim=yes, merge=no)'

Thank you for any advice.

Mike Fitzpatrick wrote on Mar 24, 2012

Two questions: 1) Which architecture is this? And b) if you run the IMCOPY command by itself does it work, e.g.


imcopy ("b0212[0]", "foo.fits", verbose=no)

do you see the same error?

The V2.16 REPO is full but I haven't yet had time to verify that all the packages were relinked yet.

Robert Janusz wrote on Mar 24, 2012

fitz

Two questions: 1) Which architecture is this? And b) if you run the IMCOPY command by itself does it work, e.g.
imcopy ("b0212[0]", "foo.fits", verbose=no)
do you see the same error?

The V2.16 REPO is full but I haven't yet had time to verify that all the packages were relinked yet.

I use the 32-bit linux architecture installed from "iraf-linux.tar.gz".
When I run:

mscred> imcopy ("b0212[0]", "foo.fits", verbose=no)

I get:

Cannot open file (b0212)

so the problem remains. But I have an image "vbias" created with the v. 2.15 with "ccdproc ... merge+" and I can "imcopy" it without any problem. So I am in trouble only with MEF files.

Mike Fitzpatrick wrote on Mar 24, 2012

It seems to want the ".fits" extension explicitly, I'm looking at it .....

Robert Janusz wrote on Mar 24, 2012

When I put:

imcopy ("b0212.fits", "foo.fits", verbose=no)

I get:

ERROR: FXF: must specify which FITS extension (b0212.fits)

and the code:

mscred> imcopy ("b0212.fits[0]", "foo.fits", verbose=no)

gives the "foo.fits" but this is an image header inside, the same as:

imhead b0212.fits[0] lo+

Mike Fitzpatrick wrote on Mar 24, 2012

The quick workaround is to do


cl> reset use_new_imt = no 
cl> flpr 0                     (zero, not oh)

to defeat the new image expansion code and get back the old behavior, but otherwise your last report is what I would expect from v215 as well. Note you can also change the default setting of 'use_vo' in the hlib$zzsetenv.def file

Robert Janusz wrote on Mar 24, 2012

Yes, the "quick workaround" works well and the code:

ccdproc b0212 overs+ trim+ merge+

works without any problem. Thank you so far.

Mike Fitzpatrick wrote on Mar 24, 2012

The bug has been fixed and the distribution files were updated with the patch. I was planning to wait a few days to see what else is reported before building a proper patch update, for now I hope you won't mind downloading the v216 installation file and unpacking it once more. Thanks for the report.

Robert Janusz wrote on Mar 24, 2012

Unpacked new "iraf-linux.tar.gz" and I get:

mscred> ccdlist b
b[im1][1036,2032][ushort][][1][]:SYM6712
b[im2][1036,2032][ushort][][2][]:SYM6712
mscred> ccdproc b
b[im1]: Mar 25 13:20 Trim is [9:1016,9:2024]
b[im1]: Mar 25 13:20 Overscan is [1020:1035,9:2024], mean 1798.776
b[im2]: Mar 25 13:20 Trim is [9:1016,9:2024]
b[im2]: Mar 25 13:20 Overscan is [1020:1035,9:2024], mean 1798.086
ERROR: Cannot open file (tmp$ccdproc4264eP)
  "sigscale=0.1, pclip=-0.5, grow=0., verbose=no)"
     line 417: mscsrc$ccdproc.cl
     called as: `ccdproc (images=b)'
mscred> ccdlist tmp$ccdproc4264eP
tmp$ccdproc4264eP[im1][1008,2016][real][object][1][][OT]:SYM6712
tmp$ccdproc4264eP[im2][1008,2016][real][object][2][][OT]:SYM6712

So I make:

mscred> reset use_new_imt = no 
mscred> ccdproc b
b[im1]: Mar 25 13:25 Trim is [9:1016,9:2024]
b[im1]: Mar 25 13:25 Overscan is [1020:1035,9:2024], mean 1798.776
b[im2]: Mar 25 13:25 Trim is [9:1016,9:2024]
b[im2]: Mar 25 13:25 Overscan is [1020:1035,9:2024], mean 1798.086
b: Merge amplifiers

And it worked. Still problems.

Mike Fitzpatrick wrote on Mar 24, 2012

Could I ask you to please download the system once more? If its the problem I think it is then it should be fixed, otherwise I need to get some data so I can reproduce the problem. Thanks.

Robert Janusz wrote on Mar 24, 2012

I downoladed today a new 32bin linux version, installed it, and the problem remains. I also tried download the example image into

anonymous@iraf.noao.edu/pub

but it was not possible, because I do not have sufficient permission to do it.[/code]

Francisco Valdes wrote on Mar 24, 2012

Hi,

I used some dual-amp Mosaic data and tried to reproduce a problem with the latest V2.16. So could you please put an problem image in the ftp pub area and let me know. Also the parameter listing from "lpar mscred ccdproc" and any thing else you think might be appropriate, such as if you have a personalized instrument file.

Thanks,
Frank Valdes

Robert Janusz wrote on Mar 24, 2012

I was not able to ftp it, so I put the data on my server and wrote a private message to you on the iraf.net and an e-mail to the "Contat Us" how to download the problematic image. Thank you,
rj

Mike Fitzpatrick wrote on Mar 24, 2012

Thanks for the data and the detailed log, I was able to reproduce the problem and found that aside from the earlier problem in the template expansion, the MSCRED package was accidently relinked with a pre-release version of the system containing a second bug. These have both now been fixed and verified and I was able to run your data with no problem without having to use the workaround.

You'll need to re-install both IRAF and MSCRED. I've built the IRAF patch but not the update script yet so simply get the new distribution file once again (sorry for the repeat installs). To update MSCRED (and all other packages), just do


% cd $iraf/extern
% make update

Thanks again for your report, and your patience.

Robert Janusz wrote on Mar 24, 2012

I dowloaded the file "iraf.lnux.x86.tar.gz" from a link of:

http://iraf.noao.edu/

and it did not work - the situation was the same. So I opened:

ftp://iraf.noao.edu/iraf/v216/PCIX/

and found there "iraf.lnux.x86.tar.gz" (191660 KB, 03/23/2012, 01:25:00 AM) and "iraf.lnux.x86.gz" (195880 KB, 03/28/2012, 08:04:00 PM). I dowloaded the second one, made the installation and I was able to get the correct result. Thank you.

The problem is with PyRAF:

PyRAF 1.11 Copyright (c) 2002 AURA
Python 2.6.6 Copyright (c) 2001-2010 Python Software Foundation.
Python/CL command line wrapper
  .help describes executive commands
--> mscred
...
--> displ a[1] 1
z1=1794.741 z2=1829.
--> displ a[2] 1
z1=1784.413 z2=1839.642
--> ccdlist a
a[im1][1036,2032][ushort][][1][]:SYBIA_1
a[im2][1036,2032][ushort][][2][]:SYBIA_1
--> ccdproc a
Instrument (? for a list) ('instrmsc'): 
Warning: Cannot open file (instrmsc)
Killing IRAF task `imheader'
Traceback (innermost last):
  File "<CL script CL1>", line 1, in <module>
  File "<CL script mscred.ccdproc>", line 192, in ccdproc
    if (iraf.imaccess(Vars.outname + '[0]') == yes):
IrafError: Error running IRAF task imheader
IRAF task terminated abnormally
ERROR (741, "Cannot open file (tmp$ccdproc12836cN)")

Robert Janusz wrote on Mar 24, 2012

The file

ftp://iraf.noao.edu/iraf/v216/README.VO

has no data.

Mike Fitzpatrick wrote on Mar 24, 2012

The README.VO file is still in preparation, I just tend to create empty files so people will notice and nag me until I actually write it 8-)

and found there "iraf.lnux.x86.tar.gz" (191660 KB, 03/23/2012, 01:25:00 AM) and "iraf.lnux.x86.gz" (195880 KB, 03/28/2012, 08:04:00 PM). I dowloaded the second one,

Ooops, the build script was still using the old naming scheme. Using the "second file" you mention is correct.

The problem is with PyRAF:

I think the problem begin with the message about not finding the 'instrmsc' instrument setting, does defining this help? I think the rest of the trace is bogus since IMHEAD is never called directly from CCDPROC. I verified that the imaccess() function works properly without the ".fits" extension as well.

Some of the new v2.16 features certainly won't work under PyRAF unless/until it is updated, but otherwise things should generally be backwards compatible.

Robert Janusz wrote on Mar 24, 2012

Not exactly. The extern "mscred.ccdproc.cl" in line 103 has:

tmpfname = _ccdlist.ccdproc; _ccdlist.ccdproc="setinstrument"

really unexpected instruction here.

Robert Janusz wrote on Mar 24, 2012

The newest IRAF of today does not process MEF files:

j0062[im1]: Apr 13 22:03 Trim is [9:1016,9:2024]
j0062[im1]: Apr 13 22:03 Overscan is [1020:1035,9:2024], mean 1726.488
j0062[im2]: Apr 13 22:03 Trim is [9:1016,9:2024]
j0062[im2]: Apr 13 22:03 Overscan is [1020:1035,9:2024], mean 1748.158
j0062: Merge amplifiers
ERROR: segmentation violation
  "imdelete (mergeinput, verify-)"
     line 442: mscsrc$ccdproc.cl
     called as: `ccdproc (images=j0062, ccdtype=, overscan=yes, trim=yes, zerocor=no, darkcor=no, flatcor=no, merge=yes)'
     called as: `cl ()'
     called as: `cl ()'
     called as: `cl ()'
     called as: `cl ()'
     called as: `cl ()'
     called as: `cl ()'
     called as: `cl ()'
     called as: `cl ()'
  "clbye()"
     line 41: fitsutil$fitsutil.cl
     called as: `fitsutil ()'
  "fitsutil"
     line 5: mscred$mscred.cl
     called as: `mscred ()'
     called as: `cl ()'
  ""ccdproc(s1,ccdtype='',overs+,trim+,zeroc-,darkc-,flatc-,merge+)\n") ..."

The commands:

reset use_new_imt = no
flpr 0

do not help.

Mike Fitzpatrick wrote on Mar 24, 2012

I verified that the files in the archive still process your test image from the earlier report, but I'm wondering whether the problems you had with the update might mean the files aren't installed correctly. Can you verify that binaries for the core system and the MSCRED package date from at least April 11 ??

Robert Janusz wrote on Mar 24, 2012

Today I did:

% cd $iraf
% make latest

and the output is:

Updating IRAF v2.16 to latest release.

Tue Apr 17 13:49:12 CEST 2012

(util/iraf_update -all)
Updating utility scripts ....Done
Updating IRAF system  ....
iraf_latest: cwd =  /iraf/iraf
Updating binaries:   linux
Downloading: patch-src.tar.gz Unpacking ... Done.
Downloading: patch.lnux.x86.tar.gz Unpacking ... Done.
Update all external packages ...
make[1]: Entering directory `/iraf/iraf/extern'
Initializing repository data ....
Creating system makefile ....
Setup Complete.
Updating repository data ....
Package adccdrom     is not installed ....
Package cfh12k       is not installed ....
Package ctio         is not installed ....
Package deitab       is not installed ....
Package esowfi       is not installed ....
Package fitsutil     is current ....
Package guiapps      is not installed ....
Package mem0         is not installed ....
Package mscdb        is not installed ....
Package mscred       is current ....
Package mtools       is not installed ....
Package nfextern     is not installed ....
Package optic        is not installed ....
Package patch        is not installed ....
Package rvsao        is not installed ....
Package song         is not installed ....
Package sqiid        is not installed ....
Package stsdas       is current ....
Package tables       is current ....
Package ucsclris     is not installed ....
Package upsqiid      is not installed ....
Package xdimsum      is not installed ....
Done
make[1]: Leaving directory `/iraf/iraf/extern'

Update Status:    Successful

Tue Apr 17 13:52:22 CEST 2012

Now I go to "cl" and make "setinstr" then:

mscred> ccdlist a
a[im1][1036,2032][ushort][][1][]:SYBIA_1
a[im2][1036,2032][ushort][][2][]:SYBIA_1
mscred> ccdproc a
a[im1]: Apr 17 13:55 Trim is [9:1016,9:2024]
a[im1]: Apr 17 13:55 Overscan is [1020:1035,9:2024], mean 1810.755
a[im2]: Apr 17 13:55 Trim is [9:1016,9:2024]
a[im2]: Apr 17 13:55 Overscan is [1020:1035,9:2024], mean 1807.158
a: Merge amplifiers
ERROR: segmentation violation
  "imdelete (mergeinput, verify-)"
     line 442: mscsrc$ccdproc.cl
     called as: `ccdproc (images=a)'

Then I make:

mscred> fcache list
mscred> tinfo ds9.xml 
ERROR: Cannot open file (cache$.f775517028)
mscred> tinfo http://iraf.noao.edu/votest/usno-b.xml
ERROR: Cannot open file (cache$.f2018547945)

Then I made:

% cd iraf/extern
% pwd
/iraf/iraf/extern
% make mscred
Setting architecture: 'linux' .... 
Adding dependency 'fitsutil' ....
Installing package 'fitsutil' ....  [OK]
Installing package 'mscred' ....  [OK]

Then "mkiraf", "cl", "mksred", "setinstr..." and I get again:

mscred> ccdlist a
a[im1][1036,2032][ushort][][1][]:SYBIA_1
a[im2][1036,2032][ushort][][2][]:SYBIA_1
mscred> ccdproc a
a[im1]: Apr 17 14:37 Trim is [9:1016,9:2024]
a[im1]: Apr 17 14:37 Overscan is [1020:1035,9:2024], mean 1810.755
a[im2]: Apr 17 14:37 Trim is [9:1016,9:2024]
a[im2]: Apr 17 14:37 Overscan is [1020:1035,9:2024], mean 1807.158
a: Merge amplifiers
ERROR: segmentation violation
  "imdelete (mergeinput, verify-)"
     line 442: mscsrc$ccdproc.cl
     called as: `ccdproc (images=a)'

So I did a step back installing the version 2.15.1a, then:

ccdlist a
a[im1][1036,2032][ushort][][1][]:SYBIA_1
a[im2][1036,2032][ushort][][2][]:SYBIA_1
mscred> ccdproc a
a[im1]: Apr 17 15:21 Trim is [9:1016,9:2024]
a[im1]: Apr 17 15:21 Overscan is [1020:1035,9:2024], mean 1810.751
a[im2]: Apr 17 15:21 Trim is [9:1016,9:2024]
a[im2]: Apr 17 15:21 Overscan is [1020:1035,9:2024], mean 1807.149
a: Merge amplifiers

So this version is correct.

Mike Fitzpatrick wrote on Mar 24, 2012

Would you mind posting the dates of the binaries or otherwise verify they are at least April 11 or later?? I've tried several machines and updated in the same way I posted, however it works just fine for me.
If you installed a distribution file with multiple binaries, could you also do a "show arch" at the CL prompt to verify the architecture being used (i.e. the updates were for 'linux', is there a chance something is running 'linux64'??).

Robert Janusz wrote on Mar 24, 2012

I did the complete re-installation of the last binaries for 'linux32' and now it works fine with 'mscred' and 'ds9.xml' as well. Thank you.

Last post on Mar 24, 2012

IRAF Community Distribution

IRAF maintained by the community

IRAF 2.16 mscred ccdproc problem