
Mailing List Archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[tlug] Limits on file numbers in sort -m
- Date: Wed, 28 May 2014 10:48:23 +1000
- From: Jim Breen <jimbreen@example.com>
- Subject: [tlug] Limits on file numbers in sort -m
A real *n*x question for once.
I'm gearing up for a merging of a very large number of
sorted text files(*). Does anyone know if there is an upper
limit on how many sorted files can be merged using something
like: "sort -m *.interim > final".
Also, is it worth fiddling with the "--batch-size=NMERGE" option?
The system I'll be doing it on has masses of disk (10Tb), 8Gb of
RAM and is a quad processor.
TIA
Jim
(*) about 10k files, each about 4M lines of text.
--
Jim Breen
Adjunct Snr Research Fellow, Japanese Studies Centre, Monash University
Home |
Main Index |
Thread Index