Re: [tlug] SSD caching of 10x 10GB files on slow NAS

[Miles Colman <mcolman@example.com>]

Actually, video editing isn't far off.

The deal is that we have about 10 users, and at bad times 5 of them are postprocessing big data simultaneously (CFD sim results). When a dataset is large, one set of sim results (3 files) will be about 300 GB total. A medium sized sim result is about 30 GB if the user doesn't request too much data from the sim. Our main NAS is 2 TB but is usually at 99% capacity and some people have 2 TB usb drives sitting on their desks. I think our ethernet is 1000 MB/sec but I need to check.

I don't know if our reads are truly random-access, but here's a sample use case:
Joe User sits down at his aging CentOS 4.9 (Final) workstation to make a movie of the airflow around a Zeppelin. This involves reading 300 MB of velocity data from each time frame, from a sim result file containing 20 such frames. (In this case, frame reads go 1,2,3 but other users with more frames and smaller data per frame sometimes do 70, 80, 90... or 23400, 23512, 23624...)

Joe knows that the Zeppelin's drag is really bugging the operators, who moan about fuel burn. As such, he makes an isosurface of low pressure at the Zeppelin nose in the post-processor, and tells the post-proccesor to compute and show that iso surface for all 20 frames and save a picture of each one. The really time consuming steps here are making the isosurface, which involves looking at all the data in each time frame (170 million spatial data points, I don't know how they are indexed), and testing against a pressure value to see if that xyz location should be a member of the isosurface. Joe had a bad night and is feeling groggy and forgot to select a spatial subset of the data, so all the points in each frame get tested. This gets repeated for each frame.

Repeat 5 times and you have a picture of what our users are doing.