Treatment of missing MEF extensions in 2.16
James Turner wrote on Sep 08, 2012
In IRAF 2.15, if I try to access a FITS extension that doesn't exist, I get this:
In IRAF 2.16, I get this:
Whenever I specify a non-existent extension, 2.16 behaves as if I had not specified any extension at all. I can even do this:
Is that expected? It seems odd behaviour and I'm seeing a lot of test failures due to it (so far due to an invalidated assumption in PyRAF but I suspect there will be others).
Thanks again,
James.
ecl> imhead S20100122S0063.fits[BLAH]
S20100122S0063.fits[BLAH]: FXF: extname and/or extver value not found ('BLAH')
In IRAF 2.16, I get this:
ecl> imhead S20100122S0063.fits[BLAH]
S20100122S0063.fits: FXF: must specify which FITS extension (S20100122S0063.fits)
Whenever I specify a non-existent extension, 2.16 behaves as if I had not specified any extension at all. I can even do this:
ecl> imhead dev$pix[sci]
dev$pix[512,512][short]:m51 B 600s
Is that expected? It seems odd behaviour and I'm seeing a lot of test failures due to it (so far due to an invalidated assumption in PyRAF but I suspect there will be others).
Thanks again,
James.
James Turner wrote on Sep 08, 2012
Hi Mike,
We're working on 2.16 compatibility and wondering whether you have any ideas re. my above post from a few months ago?
We've also collected a few more examples of behaviour that has changed or seems odd in 2.16 and it would be very helpful to know which of these are intentional and whether you have any ideas to solve the others:
Thanks for any insight!
James.
We're working on 2.16 compatibility and wondering whether you have any ideas re. my above post from a few months ago?
We've also collected a few more examples of behaviour that has changed or seems odd in 2.16 and it would be very helpful to know which of these are intentional and whether you have any ideas to solve the others:
- 1. FITS extension names are now sometimes case sensitive:
imstat gsS20100223S0064_nomdf.fits[SCI]
- Under 2.14 prints results for the first SCI extension
- Under 2.16 prints results for all 34 extensions (as intended, I think)
imstat gsS20100223S0064_nomdf.fits[sci]
- Under 2.16 this behaves differently (note lower case sci):
Error reading image gsS20100223S0064_nomdf.fits ...
- Specifying no extension gives the same result, so this is the same behaviour as I reported in my original post (it thinks no extension was specified when it doesn't exist).
copy gsS20100223S0064_nomdf.fits full_copy.fits (followed by)
imcopy gsS20100223S0064_nomdf.fits[sci,2] full_copy.fits[OBJ,2,append+]
- IRAF 2.16 gives:
gsS20100223S0064_nomdf.fits -> full_copy.fits[append+]
ERROR: FXF: must specify which FITS extension
(gsS20100223S0064_nomdf.fits)
- Again, the extension is not found, but in this case the error is different
but...
=imaccess("gsS20100223S0064_nomdf.fits[sci]")
- Returns "yes" under 2.16
- Apparently imaccess is NOT case sensitive
2. Misleading errors when input extension is not found:
imcopy gsS20100223S0064_nomdf.fits[SCI,35] somefile.fits[append]
- Under 2.14 gives this error because there are only 34 SCI exts:
ERROR: FXF: extname and/or extver value not found ('SCI,35')
- Under 2.16 gives a less clear error (as illustrated above when
the wrong case is used):
gsS20100223S0064_nomdf.fits -> somefile.fits[append]
Error (1113): FXF: must specify which FITS extension
(gsS20100223S0064_nomdf.fits)
- This also seems to be due to the change I reported in my first post.
imhead btrg2N20111206S0387.fits[SCI,19]
- IRAF 2.14 gave:
ERROR: FXF: extname and/or extver value not found ('SCI,19')
- IRAF 2.16 sometimes gives:
Error (502): integer divide by zero
- This error isn't repeatable every time; sometimes it gives the same "must specify which FITS extension" message as above. It usually occurs following another error. This particular file is still proprietary, so we'll try to find another example if you want to look at it.
3. Change in append behaviour:
copy gsS20100223S0064_nomdf.fits full_copy.fits (followed by)
imcopy gsS20100223S0064_nomdf.fits[SCI,1] full_copy.fits[OBJ,1,append+]
- IRAF 2.14 appends the extension as OBJ,1 at the end
- IRAF 2.16 now fails to append this:
Error (1112): FXF: extname and/or extver value already exists in extension (1)
- There was no OBJ,1 already, however (only SCI extensions).
- With the "dupname" kernel parameter it successfully appends a duplicate SCI,1 instead of the requested OBJ,1 (previously dupname was only necessary when specifying identical EXTNAME & EXTVERs).
- Mark thinks it's checking the output file for the specified input extension name (SCI,1) rather than the specified output extension name (OBJ,1).
4. Different odd behaviour when the above FITS file has a binary table as its first extension (before the 34 SCI image extensions):
imstat gsS20100223S0064.fits[SCI,35] or imstat gsS20100223S0064.fits[sci]
(neither extension is found)
- Under 2.16 this gives (as if reading an empty extension):
# IMAGE NPIX MEAN STDDEV MIN MAX
gsS20100223S0064.fits 0 INDEF INDEF INDEF INDEF
- The other file gives:
Error reading image btrg2N20111206S0387.fits ...
or sometimes the divide by zero error mentioned above.
Thanks for any insight!
James.
James Turner wrote on Sep 08, 2012
By the way, we were comparing against 2.14 just for convenience, but I believe these differences are all new since 2.15 (otherwise we'd have picked them up before).
Mike Fitzpatrick wrote on Sep 08, 2012
Sorry for the slow reply to this. There appear to be several issues but at first glance they all appear to be related to the new image template code in v2.16. These will be fixed for the next release but in the meantime the workaround is to do
to revert to the old template code. You can put this in your package loading script to avoid it for your tasks entirely, or else edit the hlib$zzsetenv.def file to disable it for all tasks.
The one question I need to this about is the behavior in your second message where specifying an extension would previously give only the first instance, but now lists them all. The new template code is meant to allow for easy expansion into lists but I think you'd only want the second behavior when you explicitly use the '@' operator, and so the default should still produce only one extension. If you agree then that'll be how I fix it.
Note that patched binaries aren't possible as a workaround since this code is compiled into all tasks, but it will be fixed for the next update. Thanks for the report.
cl> reset use_new_imt = noto revert to the old template code. You can put this in your package loading script to avoid it for your tasks entirely, or else edit the hlib$zzsetenv.def file to disable it for all tasks.
The one question I need to this about is the behavior in your second message where specifying an extension would previously give only the first instance, but now lists them all. The new template code is meant to allow for easy expansion into lists but I think you'd only want the second behavior when you explicitly use the '@' operator, and so the default should still produce only one extension. If you agree then that'll be how I fix it.
Note that patched binaries aren't possible as a workaround since this code is compiled into all tasks, but it will be fixed for the next update. Thanks for the report.
James Turner wrote on Sep 08, 2012
Thanks, we'll disable use_new_imt in our package for now then.
Thinking about your question, it does seem a good idea to require @ in order to expand out file.fits[SCI] and otherwise only provide one extension, for backwards compatibility.
Cheers,
James.
Thinking about your question, it does seem a good idea to require @ in order to expand out file.fits[SCI] and otherwise only provide one extension, for backwards compatibility.
Cheers,
James.
Last post on Sep 08, 2012