Re: tlug: Interesting new server

[Karl-Max Wagner <karlmax@example.com>]

> This is definitely an interesting project and one of the
> members of the Beowulf project (might even be the leader)
Don Becker is the leader of the Beowulf project. He is also the
one who wrote a large portion of the ethernet drivers for Linux.
> myself, but nevertheless, I think this announcement is a bit
> too optimistic.  This is for a hardware and a software
Not necessarily. Beowulf has been around for a while. In fact,
there are already a hundred Beowulf clusters in daily use
already. One of the biggest so far is Los Alamos National
Labratory's Avalon with 70 500 MHz DEC Alpha machines.
> reason.  Hardware: Even a 100MBit Ethernet is a joke
> compared to the networks used in `real' parallel machines.
Super high communications speed is not necessarily a
prerequisite for paralell processing. I'll explain later why.
> ....<Hardware considerations>..... 
> Software: Unfortunately, here Linux is the problem.
> ......<Networking considerations>.....
In a paralell processing system several strategies exist - but a
common objective is to keep processor intercommunications at a
strict minimum, because otherwise communications is using up
considerable computing power - no matter how clever and
efficient the intercommunications strategy is. There is another
problem: in larger clusters routing between the nodes is a major
problem. Thus simple communications strategies arrive quickly at
their limits. The use of TCP/IP, resource eating as it may be in
smaller networks, is justified in larger networks where special
routers have to provide high throughput communications between
nodes, like in the Avalon system. TCP/IP has the huge advantage
of scalability to even huge systems - even systems of the size
of the Internet remain manageable that way.

Beowulf is a so called "coarsely grained" paralell computing
system, that is nodes are working pretty much autonomously and
doesn't require huge amounts of intercommunications. Actually,
the creators of Avalon were musing on the influence of faster
communications onto computing power and came to the conclusion
that not much could be gained and decided that it wasn't worth
the expense. This shows that their network is certainly not
short of actual requirements.
> Extreme Linux is surely interesting for some applications,
> but `high performance computing' is still something quite
> different.
I'd rather say it is a late reaction onto the fact that Beowulfs
lately have been popping up like mushrooms after a summer rain
and they try to serve this emerging market, too.

Avalon, BTW, made it already into the "Top 500" list of
supercomputers with 19.2 GFlops ( sustained, measured with
Linpack ). This certainly isn't just peanuts. The system cost
them arounf 150 000 US-$ and has the same performance as a SGI
Origin with 64 processors, but costing 10 times as much ! If
they have that much money they could push it to 700 nodes and
push their system maybe into the top 50 range or so....

The creation of Avalon from the arrival of the boxes until it
was up and running took under 2 weeks - proving that the Beowulf
system has production quality and isn't just an interesting
experiment any more.

My impression is that the supercomputer companies have some
reason to worry: Beowulf definitely IS eating into the low end
supercomputing market and is working its way upward - no wonder
with a price / performance advantage of about an order of
magnitude.

The next threat arrives when teaming the Beowulf nodes with
multi signal processor plug in boards to help the main processor
in number crunching and other chores. This has the power of
pushing performance by another order of magnitude without
increasing the price a lot. Hank Diez ( of PAPERS fame ) is
thinking into that direction. Expect these systems being able to
make it into the top class of supercomputing. Even more reasons
to worry for the supercomputer manufacturers.

The amazing thing is how big Linux has become: From it initial
design goal for a PC-UNIX system it is making it at an ever
increasing pace into every nook and cranny of computing -
downwards to microcontrollers and upwards into supercomputing.
It seems to me that Linux, GNU & Co. are seriously vying for
world dominion and historians in the future will regard all this
as the most important historical fact of the ending 20 th
century and the beginning of the 21st century. Maybe in 50
year's time or so free software and open circuit designs will be
commonplace and that peple by then will wonder how the world was
before this was the case - and most probably they will be unable
to understand such a world - as little as we are able to really
understand the way of thinking of a medieval knight ( just read
the High History of the Holy Graal and you know what I mean...).

                               Karl-Max Wagner
                               karlmax@example.com
--------------------------------------------------------------
Next Nomikai: 15 May Fri, 19:30 Tengu TokyoEkiMae 03-3275-3691
Next TLUG Meeting: 13 June Sat, Tokyo Station Yaesu gate 12:30
Featuring Stone and Turnbull on .rpm and .deb packages
--------------------------------------------------------------
Sponsor: PHT, makers of TurboLinux http://www.pht.co.jp