IRAF+multiCPU

Doug Tody wrote on Dec 15, 1998

Hi Enrico,

>     I have a question of a rather general scope: what does it happen
>  if we run Iraf on a multi-processor machine (eg., a DEC Alpha with
>  4 CPU) ?  A number of us here in Padua are trying to get ready to
>  reduce imaging data from big CCD arrays (8kx8k or more), so we are
>  trying to figure out how to speed up frame crunching by IRAF.
>  Can you provide any docs/advice ?  Thank you !
> 
>  Best regards,    Enrico Held 

We have a similar problem here with our 8K by 8K CCD mosaic.  We reduce
the Mosaic data with IRAF (MSCRED package) on fast Ultrasparc systems with
2 cpus, 512 MB to 1 GB of memory, 80 or more GB of disk.  Given a powerful
enough machine there is no problem reducing such data.

IRAF doesn't currently make any explicit use of the multiple CPUs, e.g.
by distributing vector computation concurrently to several CPUs (this is
possible but would require special compilers and computation).  Hence the
multiple cpus won't speed up a single process.  They still help however
in most cases since overall system usage is typically multiprocess, with
several processes executing simultaneously.  For example, you could have
an image display operation and CCD reduction going simultaneously and this
alone would use 2-3 cpus.

In a multiple-CPU system the processors all share the same disk and memory.
This can be advantageous, e.g. if multiple operations are being performed
on the same data it will probably be buffered in main memory and the
operations can proceed very quickly with minimal disk i/o.  On the other
hand, most systems are limited by disk i/o, and multiprocessing can make
this situation even worse.  It is desirable to pick the disk i/o subsystem
carefully for a multiprocessing machine, and can be necessary to plan the
processing carefully to avoid excessive disk seeking.  Having a large main
memory to buffer data is essential.  A fast disk array using multiple
spindles (disks), disk striping, and a fast (e.g. fiberchannel) i/o
connection can help as well.  Due to the expense we have never been able
to try a disk array here however.

	- Doug

---

Last post on Dec 15, 1998