Re: [tlug] Re: rsync efficiency (was: The Mother of All (bash) Commands)

["david.blomberg" <david.blomberg@example.com>]

Jim Breen wrote:
> On 02/04/2008, Attila Kinali <attila@example.com> wrote:
> > On Tue, 1 Apr 2008 20:08:02 +1100
> >  "Jim Breen" <jimbreen@example.com> wrote:
> >  > Attila Kinali <attila@example.com> stunned us all with:
> >  >
> >  > >  My point is that rsync should at least be as fast as
> >  > >  tarring and scp'ing said directory tree, ...
> >  >
> >  > Why?
> >
> > If it isn't faster, what's the point in using rsync?
>
> For updating a directory full of small files - no point at all. Don't
> know why you tried. It doesn't mean that rsync is useless/hopeless/bad/
> inefficient/etc. It means you chose the wrong tool.
In my last set of test runs small file using rsync actually was much 
quicker than scp-all-files and the tar-scp-untar methods the reason the 
scp was so slow was each file hardly got up to speed on the transfer 
before starting the next file (I never even came close to getting up to 
a decent transfer rate except for the larger files).  while in rsync the 
whole thing seemed to  be happening over one established connection the 
and transfer rate stayed fairly high the whole time.  The tar-scp-untar 
well just too slow pre and post processing made it a dog.  I actually am 
moving back to using rsync after a long time of hating it.

but to each their own run some test and see what kind of results you 
get; before deciding which method to use.