Re: [tlug] Raid5 box & backup

[Curt Sampson <cjs@example.com>]

On 2008-05-26 03:58 +0200 (Mon), Christian Horn wrote:

> The journaling filesystem is another layer and stores everything in the
> container-file while nfs/smb would work filewise.

You seem to consider "extra layers" to be a more than a very, very
minimal potential source of file read/write errors, which is certainly
not the case.

Consider that the journalling filesystem has much better recovery than
NFS or SMB when a network connection is broken in the middle of a file
update.

> Indeed, nice idea, that way rsync would finish faster. The "combine
> blocks"-software had to be evailable on mac/win/linux in this case
> thou....

No, you can build a dozen block devices on the server, concatenate them
to one block device on the server, and export that one block device over
the network.

> Also rsync could probably be reworked to handle big file in breaking 
> them into pieces and checksuming/transferring if needed those.

Rsync alredy does exactly that; it's part of the whole point of rsync.
However, as I pointed out earlier, rsync has to read all of every file
with a changed timestamp, on both ends of the connection, with every
sync. If that file is a hundred gigs in size, that means at a minimum
100 GB of reads and hashes at each end. With a modern disk, assuming the
hash uses minimal CPU, that's still a good 30 minutes of clock time, not
to mention trashing the I/O performance of the system for that time.

cjs
-- 
Curt Sampson       <cjs@example.com>        +81 90 7737 2974   
Mobile sites and software consulting: http://www.starling-software.com