Re: tlug: Re: multi-processor linux configuration ?

["Andrew S. Howell" <andy@example.com>]

>>>>> "Manuel" == Manuel M T Chakravarty <chak@example.com> writes:

    Manuel> "Andrew S. Howell" <andy@example.com> wrote,
    >> >>>>> "Darren" == Darren Cook <darren@example.com> writes:
    >> 
    Darren> One interesting thing I read - the more machines you have,
    Darren> the more often you'll seem to have machine failures. So
    Darren> make sure the machines are easy to replace, and that your
    Darren> application can adapt to machines coming and going.
    >>  I would just setup one master machine and then use rdist to
    >> keep them all in sync. Keeping all the file systems ( partions
    >> ) the same will make life a lot simpler. You should be able to
    >> do the install of a new machine over then local net. Once a
    >> very basic system is installed, rdist from the master machine
    >> and your set.

    Manuel> I'd use `rsync':

    Manuel>   http://samba.anu.edu.au/rsync/

    Manuel> Saves a lot of network traffic in case you apply some
    Manuel> changes, ie., only the changes are transferred (like with
    Manuel> diff).

I tried it once. It used so much CPU that I gave up. 

If I were really worried about bandwidth, I think I design something
different. I would have the master machine broadcast the time-stamps
of each file in the tree and have the clients send back a list of
which ones they are interested in. After receiving the replies for all
the clients, I would then broadcast out the files. The individual
client machines would then pick off the files they are interested in.

Having said that, we have used rdist on large trading floors ( several
hundred machines ) with good results. This is usually done at night,
so the fact that it takes a couple hours is not a problem.

If you are updating just one machine, such as when adding a new one,
you can do the rdist only to that one machine.

For a set of machines that are usually in sync with one another, the
number of files that change tend to be rather small. More bandwidth is
probably used in figuring out what to send. One could probably speed
up rdist a bit by having the master create a file containing the
structure and time-stamps of the tree it wants to propagate, send that
to the clients ( compressed ), and have the clients make requests for
the files from the server. As rdist is now, it walks down the tree and
sends the time-stamps to the client, which then decides if it needs
that file or not( I think... ). Over a slow link, that part of the
process could consume a bit of time.

Sorry to ramble on....

Andy
---------------------------------------------------------------
Next Nomikai: 20 November, 19:30 Tengu TokyoEkiMae 03-3275-3691
Next Meeting: 12 December, 12:30 Tokyo Station Yaesu central gate
---------------------------------------------------------------
Sponsor: PHT, makers of TurboLinux http://www.pht.co.jp