problems with pixeltypes in ccdred

Paco Garzón wrote on Oct 18, 2007

Hi all,

I'm trying to reduce a bunch of images using ccdred, of which I'm used to. Since this time the number of files is exceedingly large, I did try to reduce storage space by setting the output pixtype to ushort, which is indeed the original type of the files as they are output from the ccd camera (bitpix=16, bscale=1, bzero=32768). You can read from the following list how when saving as ushort the files are not properly written after just an overscan correction for example, as demonstrated by the imstat results.

Am I doing something wrong?

Thanks,

Paco
-----------------------------
ccdred> ccdlist *.fits
B220206_0007.fits[2148,2048][ushort][zero][NII_IAC41]:

ccdred> lpar ccdred
(pixeltype = "ushort real") Output and calculation pixel datatypes
(verbose = yes) Print log information to the standard output?
(logfile = "logfile") Text log file
(plotfile = "") Log metacode plot file
(backup = "") Backup directory or prefix
(instrument = "/scratch/TAI/ccdiac80.dat") CCD instrument file
(ssfile = "subsets") Subset translation file
(graphics = "stdgraph") Interactive graphics output device
(cursor = "") Graphics cursor input
(version = "2: October 1987")
(mode = "ql")

ccdred> ccdproc im=B220206_0007 out=C220206_0007
B220206_0007: Oct 18 11:54 Overscan section is [1:50,1:2048] with mean=512.6867

ccdred> ccdlist *.fits
B220206_0007.fits[2148,2048][ushort][zero][NII_IAC41]:
C220206_0007.fits[2148,2048][ushort][zero][NII_IAC41][O]:

ccdred> imstat *.fits
# IMAGE NPIX MEAN STDDEV MIN MAX
B220206_0007.fits 4399104 509.7 3.453 488. 566.
C220206_0007.fits 4399104 35416. 32658. 0. 65535.

ccdred> imdel C*

ccdred> lpar ccdred
(pixeltype = "real real") Output and calculation pixel datatypes
(verbose = yes) Print log information to the standard output?
(logfile = "logfile") Text log file
(plotfile = "") Log metacode plot file
(backup = "") Backup directory or prefix
(instrument = "/scratch/TAI/ccdiac80.dat") CCD instrument file
(ssfile = "subsets") Subset translation file
(graphics = "stdgraph") Interactive graphics output device
(cursor = "") Graphics cursor input
(version = "2: October 1987")
(mode = "ql")

ccdred> ccdproc im=B220206_0007 out=C220206_0007
B220206_0007: Oct 18 11:56 Overscan section is [1:50,1:2048] with mean=512.6867

ccdred> ccdlist *.fits
B220206_0007.fits[2148,2048][ushort][zero][NII_IAC41]:
C220206_0007.fits[2148,2048][real][zero][NII_IAC41][O]:

ccdred> imstat *.fits
# IMAGE NPIX MEAN STDDEV MIN MAX
B220206_0007.fits 4399104 509.7 3.453 488. 566.
C220206_0007.fits 4399104 -2.997 3.453 -24.68 53.32

---

Francisco Valdes wrote on Oct 18, 2007

While the feature was included in the program many years ago when disk spaces were much smaller, it was never very throughly tested. While there may be some way to make it work as expected I recommend you don't try and use the short data type during processing. To then save disk space you would compress your data by possibly converting to short (or scaling and converting) using chpixtype. In addition, particularly if you convert to short, you could use a standard compression utility such as gzip.

Note that converting data to short requires some scientific evaluation of the data. Mainly it depends on the dynamic range of the data and the noise. When converting to short you want to have the noise, usually taken as the noise at the background level, sampled by at least 10 times; i.e. a digital unit <= sigma/10.

I'm sorry I can't tell you how to use the "short" processing option but the most time efficient thing is to tell you not to depend on it.

Frank Valdes

---

Last post on Oct 18, 2007