Re: [tlug] how to tune reiser4 for millions of files?

[Michal Hajek <hajek1@example.com>]

thank you for a quick response, I will try to clarify the issue...

* Curt Sampson (cjs@example.com) [100128 11:00]:
> I can provide more ideas if you tell us more about what sort of analysis
> you're trying to do.

the analysis itself is not a problem at the moment. I believe that
by rewriting the program one can compute the whole thing in an hour or
so. Especially I believe one could employ cuda and nvidia card to get
even better result, since the thing is easily paralelizable (? not sure
about the correct English word). 

My attention is more on the hw or system side of the problem. That is, 
can I do something with the system (OS, hw..etc.) to speed things up?

My first idea was to use reiser4, since I have found this info [1]:

"Fast handling of very large directories with hundreds of millions of
files (yes, millions of files in a single directory without affecting
performance)." 

Later I found a tip about using "notail" mount option or maybe a
different hash than the default "r5". But this I did not try yet. 
Does anyone have an experience?

All maybe a different filesystem altogether?  Or maybe using lvm with
the idea to "spread" the fs onto more hw, or maybe use ssd disk? Or...I
do not know. So maybe somebody comes with an interesting idea :]

The kernel is 2.6.31-gentoo-r6, hw is  dell poweredge 6650 [2]
with two scsi Megaraid PERC 4/dc cards. Right now I perform the analysis
on one array with raid5. The other array is free. 

Surely, the application will get rewritten and if I do not find any good
idea to speed the system, it will be no catastrophe too. But I am very
curios if there is some tip or tweak which would help at least a little
bit...

best regards
Michal

[1] http://www.ibm.com/developerworks/aix/library/au-unix-reiserFS/
[2] http://www.dell.com/downloads/global/products/pedge/en/66x0_specs.pdf
    16GB ram, 4x intel 2.4 GHz hyperthreading cpu.