iraf-v216 · Code · Issues (50) · Pull requests (81)
iraf.net Issue #53
listpix doesn’t work with sections
closed olebole opened this issue on 2017-05-10 · 16 comments
olebole commented on 2017-05-10
I followed the file testproc.ps.Z for some basic tests. There, the listpix command does not work as expected:
ecl> listpix dev$pix[300:305, 200:205]
ERROR: singular matrix
If I omit section [300:305, 200:205], I get a result. The other use of sections in the test procedure works, but then again plotting doesn’t work:
ecl> imcopy dev$pix[300:305,200:205] image.sect
dev$pix[300:305,200:205] -> image.sect
ecl> imhead image.sect
image.sect[6,6][short]: m51 B 600s
ecl> listpix image.sect
Warning: setting LTM2_2 to 1.
ERROR: singular matrix
iraf commented on 2017-05-10
Works for me, linux and OSX. ‘Singular matrix’ sounds like an issue in
MWCS.
On Wed, May 10, 2017 at 2:21 PM, Ole Streicher wrote:
[…]
olebole commented on 2017-05-10
So it comes from the WCS? But there is none in the image? An image copied without section has the following header:
SIMPLE = T / Fits standard
BITPIX = 16 / Bits per pixel
NAXIS = 2 / Number of axes
NAXIS1 = 512 / Axis length
NAXIS2 = 512 / Axis length
EXTEND = F / File may contain extensions
ORIGIN = 'NOAO-IRAF FITS Image Kernel July 2003' / FITS file originator
DATE = '2017-05-10T21:34:10' / Date FITS file was generated
IRAF-TLM= '2017-05-10T21:34:10' / Time of last modification
OBJECT = 'm51 B 600s' / Name of the object observed
IRAF-MAX= 1.993600E4 / DATA MAX
IRAF-MIN= -1.000000E0 / DATA MIN
CCDPICNO= 53 / ORIGINAL CCD PICTURE NUMBER
ITIME = 600 / REQUESTED INTEGRATION TIME (SECS)
TTIME = 600 / TOTAL ELAPSED TIME (SECS)
OTIME = 600 / ACTUAL INTEGRATION TIME (SECS)
DATA-TYP= 'OBJECT (0)' / OBJECT,DARK,BIAS,ETC.
DATE-OBS= '05/04/87' / DATE DD/MM/YY
RA = '13:29:24.00' / RIGHT ASCENSION
DEC = '47:15:34.00' / DECLINATION
EPOCH = 0.00 / EPOCH OF RA AND DEC
ZD = '22:14:00.00' / ZENITH DISTANCE
UT = ' 9:27:27.00' / UNIVERSAL TIME
ST = '14:53:42.00' / SIDEREAL TIME
CAM-ID = 1 / CAMERA HEAD ID
CAM-TEMP= -106.22 / CAMERA TEMPERATURE, DEG C
DEW-TEMP= -180.95 / DEWAR TEMPRATURE, DEG C
F1POS = 2 / FILTER BOLT I POSITION
F2POS = 0 / FILTER BOLT II POSITION
TVFILT = 0 / TV FILTER
CMP-LAMP= 0 / COMPARISON LAMP
TILT-POS= 0 / TILT POSITION
BIAS-PIX= 0 /
BI-FLAG = 0 / BIAS SUBTRACT FLAG
BP-FLAG = 0 / BAD PIXEL FLAG
CR-FLAG = 0 / BAD PIXEL FLAG
DK-FLAG = 0 / DARK SUBTRACT FLAG
FR-FLAG = 0 / FRINGE FLAG
FR-SCALE= 0.00 / FRINGE SCALING PARAMETER
TRIM = 'Apr 22 14:11 Trim image section is [3:510,3:510]'
BT-FLAG = 'Apr 22 14:11 Overscan correction strip is [515:544,3:510]'
FF-FLAG = 'Apr 22 14:11 Flat field image is Flat1.imh with scale=183.9447'
CCDPROC = 'Apr 22 14:11 CCD processing done'
AIRMASS = 1.08015632629395 / AIRMASS
HISTORY 'KPNO-IRAF'
HISTORY '24-04-87'
HISTORY 'KPNO-IRAF' /
HISTORY '08-04-92' /
while the one with sections adds
WCSDIM = 2
LTV1 = -299.
(and changes NAXIS1, NAXIS2, DATE, IRAF-TLM accordingly),
iraf commented on 2017-05-10
Pixel coords are a form of WCS. See sys$mwcs/mwlu.x for your error message
and trace it backwards.
On Wed, May 10, 2017 at 2:42 PM, Ole Streicher wrote:
[…]
olebole commented on 2017-05-10
IMO the section routine writes an incomplete WCS: it writes a WCSDIM, but no further WCS info. And this is then not taken as a unity matrix, but as a zero one.
iraf commented on 2017-05-10
Keywords aren’t necessarily written when they are the default value anyway
(typically zero values). This has worked for 20+ years this way so you’ve
changed something ……
On Wed, May 10, 2017 at 2:55 PM, Ole Streicher wrote:
[…]
RobSteele49 commented on 2017-05-11
Just for clarification which version of IRAF and which OS are you seeing this error. Is there any chance that the old version of testproc might not have the correct syntax?
olebole commented on 2017-05-11
@RobSteele49 the syntax didn’t change. OS is Linux.
@iraf All changes are documented as pull requests here – the ones I proposed, and #39. There is nothing that could influence the sys/mws subdir – none of the PRs touches this. These are the changes between the current master branch and my work with respect to the sys subdir.
- adjusting the symlinks to be relative (#33):
sys/osb/bytmov.c->../../unix/as/bytmov.cinstead of/iraf/iraf/unix/as/bytmov.csys/osb/d1mach.f->../../unix/hlib/d1mach.finstead of/iraf/iraf/unix/hlib/d1mach.fsys/osb/i1mach.f->../../unix/hlib/i1mach.finstead of/iraf/iraf/unix/hlib/i1mach.fsys/osb/r1mach.f->../../unix/hlib/r1mach.finstead of/iraf/iraf/unix/hlib/r1mach.f
- regenerated several
acht*.xfiles. I didn’t remove then, but they are still newly generated during the build (contrary to what you said).
diff --git a/sys/vops/ak/achtid.x b/sys/vops/ak/achtid.x
index d030ef99..4b3cf5b5 100644
--- a/sys/vops/ak/achtid.x
+++ b/sys/vops/ak/achtid.x
@@ -12,6 +12,6 @@ int npix
int i
begin
- do i = npix, 1, -1
+ do i = 1, npix
b[i] = a[i]
end
diff --git a/sys/vops/ak/achtix.x b/sys/vops/ak/achtix.x
index 7413c08a..41ed7f22 100644
--- a/sys/vops/ak/achtix.x
+++ b/sys/vops/ak/achtix.x
@@ -12,6 +12,6 @@ int npix
int i
begin
- do i = npix, 1, -1
+ do i = 1, npix
b[i] = complex(real(a[i]),0.0)
end
diff --git a/sys/vops/ak/achtld.x b/sys/vops/ak/achtld.x
index a67a5a42..9193d3bb 100644
--- a/sys/vops/ak/achtld.x
+++ b/sys/vops/ak/achtld.x
@@ -12,6 +12,6 @@ int npix
int i
begin
- do i = npix, 1, -1
+ do i = 1, npix
b[i] = a[i]
end
diff --git a/sys/vops/ak/achtlx.x b/sys/vops/ak/achtlx.x
index ecfc2f68..4d80ae2d 100644
--- a/sys/vops/ak/achtlx.x
+++ b/sys/vops/ak/achtlx.x
@@ -12,6 +12,6 @@ int npix
int i
begin
- do i = npix, 1, -1
+ do i = 1, npix
b[i] = complex(real(a[i]),0.0)
end
diff --git a/sys/vops/ak/achtri.x b/sys/vops/ak/achtri.x
index 38b137bf..423b79b0 100644
--- a/sys/vops/ak/achtri.x
+++ b/sys/vops/ak/achtri.x
@@ -12,6 +12,6 @@ int npix
int i
begin
- do i = 1, npix
+ do i = npix, 1, -1
b[i] = a[i]
end
diff --git a/sys/vops/ak/achtrl.x b/sys/vops/ak/achtrl.x
index fa30f59c..41948e6b 100644
--- a/sys/vops/ak/achtrl.x
+++ b/sys/vops/ak/achtrl.x
@@ -12,6 +12,6 @@ int npix
int i
begin
- do i = 1, npix
+ do i = npix, 1, -1
b[i] = a[i]
end
So, unless one of these changes caused the problem, it may be something like a more modern compiler – changed zeroing of local variables or so.
olebole commented on 2017-05-11
The added header
WCSDIM = 2
LTV1 = -299.
is different from what is produced by the binary x_images.e:
WCSDIM = 2
LTV1 = -299.
LTV2 = -199.
LTM1_1 = 1.
LTM2_2 = 1.
WAT0_001= 'system=physical'
WAT1_001= 'wtype=linear'
WAT2_001= 'wtype=linear'
so one problem seems to be that the generated LTV information is incomplete.
However, if I read this file (generated by the binary x_images.e) with the self-compiled x_images.e, I still get the singular matrix, so reading in doesn’t work either.
As said before, nothing changed in mwcs between master and current version.
RobSteele49 commented on 2017-05-11
Since the master in iraf/iraf-v216 doesn’t build it seems that this is the primary problem that needs to be solved - with this item on the queue after that. I’m using gcc version gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609. I’m getting the same results my current build (branch oleboleUpdates on my fork in steelewool/iraf-v216. On that branch I’ve pulled in all of olebole’s pull requests.
I’m curious which version of the compiler iraf is using. Could you (iraf) do a gcc –version on the compiler you did the build on? That way we can compare things.
I’m assuming that the system builds for iraf when the architecture is set for linux64. Is that assumption correct?
iraf commented on 2017-05-11
On Wed, May 10, 2017 at 11:47 PM, Ole Streicher wrote:
- regenerated several acht*.x files. I didn’t remove then, but they
are still newly generated during the build (contrary to what you saidhttps://github.com/iraf-community/iraf/pull/25#issuecomment-298861155).
Because Git destroys the file dates and so the ‘ifolder’ check in the mkpkg
file fails.
diff --git a/sys/vops/ak/achtid.x b/sys/vops/ak/achtid.x index d030ef99..4b3cf5b5 100644 --- a/sys/vops/ak/achtid.x +++ b/sys/vops/ak/achtid.x @@ -12,6 +12,6 @@ int npix int i begin - do i = npix, 1, -1 + do i = 1, npix b[i] = a[i]
The loop index order is changed and so this is overwriting data when the
input/output arrays are the same. This implies you did something to change
the sizeof() data types and likely this is a 64-bit problem you introduced
affecting all tasks. Does LISTPIX work on 32-bit.
So, unless one of these changes caused the problem, it may be something like a more modern compiler – changed zeroing of local variables or so.
Keep guessing, v2.16.1 was compiled with whatever the stock linux/osx compiler was in 2013 on a recent platform.
olebole commented on 2017-05-11
As far as I understand, these files should be re-generated on a new build because they depend on the data size on the platform, right? So it is not a file-date problem – they are regenerated unconditionally.
And when looking into the data size in unix/hlib/iraf.h (symlink to iraf64.h), then the changes are as expected:
define SZ_INT 4
[...]
define SZ_DOUBLE 4
This means f.e. for ak/achtid.x (conversion int to double) that one can iterate forward, this is also how it is defined in acht.gx, and this is exactly the change shown above. The ak/acht??.x files in the git repository are thereofore from a 32-bit installation. I also pragmatically tested that using them doesn’t solve the problem.
So it is not that I magically redefined the size of the data types. The cause must be somewhere different.
Does LISTPIX work on 32-bit.
No, it doesn’t work on any data size.
olebole commented on 2017-05-12
To list the specific changes against the current master; I applied only those that are absolutely necessary to compile IRAF (BTW, it would really be nice if you could review them and merge if you think they are OK).
- #22 - Add
libc/libc.htoPATHFILESininstall - #33 - Replace absolute symlinks in
sys/osbby relative ones - #38 - Convert
mklibsetc. to/bin/shinvendor - #39 - Remove
-L/usr/lib64fromLFLAGS - #40 - Build vendor libs before starting the NOVOS build
- #45 - Call the PLT for
__sigsetjmpinstead of calling directly inzsvjmp.s - #46 - (Re-)add bin.* directory structure to
vosubdir - b95f7e0f - Remove
.BASEfiles invendor/voclient, part of #25
I don’t see that any of these changes could cause the problem.
compiler was in 2013 on a recent platform.
The object files say it was GNU C 4.4.6 20120305 (Red Hat 4.4.6-4). What if you recompile on a modern platform?
iraf commented on 2017-05-12
On Fri, May 12, 2017 at 2:33 AM, Ole Streicher > wrote:
. What if you recompile on a modern platform?
What if you recompile on a less-modern platform (you already know a GCC
version that doesn’t show the problem)? Compiler version is highly
unlikely to be the problem (based on experience), although you should be
cautious if any of the optimization levels were changed (even then,
optimizer problems are rare). If it is a compiler version, you’ll need to
track the bug to fix it. If it isn’t a compiler bug, you’ll still need to
track the bug to fix it.
olebole commented on 2017-05-12
What if you recompile on a less-modern platform (you already know a GCC version that doesn’t show the problem)?
I am trying to (specifically I installed a Debian Squeeze with gcc-4.4.5), but then I run into other build problems which I first have to fix. Maybe you could in turn try out a modern system?
although you should be cautious if any of the optimization levels were changed (even then, optimizer problems are rare)
Again (you seem to ignore me with that respect): All changes I made are well-documented (see above), and the tests are done on a clean system. I didn’t change any optimization level as I didn’t change anything that is not documented above. It would be great if you could go through this list and give your opinion whether this is acceptable (and then merge it) or what the problem with that patch is.
iraf commented on 2017-05-12
I’ll be honest: I’m not even gonna pretend I’ll look at any of this for at
least a month, I have another project release that takes priority. Merging
a trivial file deletion doesn’t really help and I don’t have time for any
more than that. So, carry on and continue to submit PRs and work on your
own branch, I’ll get back to this when I can.
On Fri, May 12, 2017 at 8:58 AM, Ole Streicher wrote:
[…]
olebole commented on 2017-05-12
Merging a trivial file deletion doesn’t really help and I don’t have time for any more than that.
The file deletion in b95f7e0f is actually required to build from source. And generally, deleting files that are not required to build the system helps others (like me) to understand the code better. And it may prevent from accidently using precompiled files instead of sources.
So, carry on and continue to submit PRs and work on your own branch, I’ll get back to this when I can.
That would be great.
Fixed in #60
Last updated on 2017-05-12