RE: LAM/MPI Parallel processing

["Jonathan Shore" <jshore@example.com>]

I did parallel processing research and used Linda and PVM (pre-MPI) in a
former life.  Parallel distributed computing is probably one of the most
interesting areas of computer science.  As for good books on parallel
processing, I'll have to check my stash.  Do you want to know more on the
theory side or on the practical implementation with MPI?

Some quick advice: 

*	localize computations
try to localize your calculations as much as possible to individual PEs
[processors] (ie do a minimum of message passing during calculations).
Sometimes this involves rethinking the algorithm or restructuring the math
so as to be able to compute partial products and recombine later.

*	intelligent node selection
If you are using machines which are being used by other people you'll want
to make sure you select based on loading, raw performance, memory profile,
etc.  Sometimes, in fact, it is faster to compute on N-1 fast boxes rather
than using N boxes where one is exceptionally slow.

*	fault tollerance
Depending on the length of your runs you might consider adding benchmarking
code which will freeze the state of your system periodically so that you can
resume should a node go down or a program crash.  Another approach is to
design your algorithm to alot tasks which can be made completely independent
of others.  If the task fails run it elsewhere.


BTW, I assume this is regarding the neuron simulation you were telling me
about at the nomikai?  To what extend does each element need to communicate
with the others?  Some problems are not appropriate for distributed
parallelism because the communication latency / bandwidth is orders of
magnitude slower than a parallel computer with an internal switching
network. 

cheers

Jonathan
E-Publishing Group K.K.

-----Original Message-----
From: simon@example.com [mailto:simon@example.com]On
Behalf Of Simon Valiquette
Sent: Monday, September 04, 2000 8:05 PM
To: tlug@example.com
Subject: LAM/MPI Parallel processing


Hi everybody.

  I'm building a simulator, which seems to work fine so far except that
it is too slow.  Since the algorithms themself cannot be significantly
more optimized, I decided with my boss to make a version that will work
on a Beowulf.

  I found out that LAM/MPI would probably be a very good mix of
efficiency and portability (the Beowulf I am building is a mix of PC,
DEC Alpha and Sun computers).

  There is a lot of documentation about using or setting up LAM/MPI, and
I find the library very easy to use.  But there is not so much good
documentation about how to make a good and efficient design for parallel
programs.  I already have some idea about how to make my simulator
working efficiently in parallel, but I'm sure that a book would be able
to give me many tips and making me saving time.

  So, I would really appreciate if someone can suggest me some good
books about designing efficient programs on a heterogeneous cluster of
computer.  I don't mind if the book use Fortran, C or any other commonly
used language (personnaly, I'm using C++), as long it help me making a
more efficient simulator, considering the available hardware.

    By the way, the structure using a master that just dispatch jobs to
slave computers are not suitable for me, because each nodes have to
communicate with some others nodes, and that the amount of data is too
big (say, easilly over one GB for a single simulation).

  Thank you in advance,

Simon Valiquette


PS:   My boss is really amazed that I can produce better quality
software using only free GPL softwares, than in using for over 1000$ in
softwares running under the other OS.

---
It took the computational power of three Commodore 64s to fly to the
moon.
It takes a 486 to run Windows 95.
Something is wrong here.

-----------------------------------------------------------------------
Special TLUG/LSWG/Project BLUE/YLUG Event:
Jon "Maddog" Hall in Japan: September 5 (Tue) 19:00        IBM Japan HQ
Next Technical Meeting: September 9 (Sat) 13:30  Place: LinuxProbe Hall
Next Nomikai Meeting: October (probably Oct 18 (Fri) 19:00)  Place: TBD
-----------------------------------------------------------------------
more info: http://www.tlug.gr.jp           Sponsor: Global Online Japan

winmail.dat