[tlug] rsync efficiency (was: The Mother of All (bash) Commands)

[Attila Kinali <attila@example.com>]

On Sat, 22 Mar 2008 18:04:56 +0900
Curt Sampson <cjs@example.com> wrote:

> On 2008-03-18 15:55 +0100 (Tue), Attila Kinali wrote:
> 
> > Yes, rsync is horribly inefficient in nearly all ways.
> > ...(ie, absolutely CPU limited).
> 
> It's actually extremely efficient for the situation it was designed for,
> which is a large amount of very similar data on two machines and a slow
> network pipe.

Unless this data is contained in a lot of small files, in which
case rsync's very stupid behaviour of traversing the file/directory
tree more than once becomes a very expensive operation.

I'm backuping my mails which are in an maildir to another machine.
Each backup takes about an hour, in which rsync does nothing
but stat files on both sites. In the end, there is about 5min
of transfering the new data.. that's all.

Of course, my example is a bit on the extreme side, but rsync
claims to be fast and efficient, which it definitly is not.

			Attila Kinali

-- 
Praised are the Fountains of Shelieth, the silver harp of the waters,
But blest in my name forever this stream that stanched my thirst!
                         -- Deed of Morred