[tlug] system (filesystem+alpha?) for off-line backups

[Kalin KOZHUHAROV <me.kalin@example.com>]

Hello TLUGgers,

In the wake of another 2TB disk failing (SMART's starting to
complain), I have been looking into my "system" for backup - a manual
off-line rotation, without specific schedule for
not-so-important-data.

Before many of you jumping and suggesting I can use the cloud and/or
specialized backup software (cow forbid hardware), NO, I don't
need/want that for my purposes.

Solving problems one at a time, my biggest issue with having a few
mostly off-line HDDs is that when I decide to plug one of the drives
in and try to look at it, it takes a lot of time to run `du -sch
/mnt/whatever/*`, or `find /mnt/whatever |grep something` and I was
wondering if by changing my choice of ext3 to something else I can
speed things up 10-100 times? Is there such a filesystem that is very
good at reading metadata that is in mainline kernel?

Or is there some user program/daemon that can cache all the metadata
(I can be OK with RO mount) so that I can search faster? Generally,
the kernel block caching performs quite well actually after the first
slow run, but working on a few large disks, it seems to get purged
quickly.

I long time ago (1990?) I remember seeing a lot of `.du` text files in
the DEC Ultrix machines that were updated by a cron job and I have
used this technique (adding .find as well), but it doesn't scale well.
Plus you need to change the way you search for information which is
PITA.
And I don't want to go database-driven apps, it will be an overkill.

So, back to the point, how do different popular filesystems score on
large amounts of data (1-2TB) and large amounts of files (1-2M)?
Is there a *nix FS that stores all the metadata as a single DB-like
structure that is easy to traverse (other than NTFS)?

I might eventually test it, once I buy a few more drives, but first
wanted to ask if anyone has direct experience.

Hoping that this will be kept practical and won't induce flame-wars,
Kalin.