GigE file server architecture

jack_aubrey · Post by **jack_aubrey** » Mon Feb 07, 2005 2:30 pm

I'm thinking of reducing noise in my computer room by moving most of my storage to a dedicated Linux file server, setting up a GigE network, and putting the file server in a closet. In the interest of cheapness I had in mind resurrecting an old AMD K6-2 350 PC and adding PCI cards to support GigE and UATA/100, with some Maxtor DM9 and Seagate 7200.7 drives backing them up.

However, I'm not sure how to get a handle on what sort of performance I could realistically achieve with this setup. I think it will primarily be limited by PCI bus contention and CPU ability to service the data transfers. The board (Tyan Trinity 100AT) only supports UATA/33 natively, which suggests it's worth using the UATA/100 card (lying around the house anyway, it came free with the Maxtor drive kit). OTOH GigE can saturate a PCI bus all by itself without contending with I/O to the drives as well. I could upgrade the CPU to a K6-2 475 for very low cost if that turns out to be a limitation.

I'm not expecting to actually get anything close to 1 gigabit performance on this, but the question is whether I can expect real-world throughput substantially greater than fast Ethernet. If not then I'll spring for a newer MB, it would just be nice to get some use out of this old box. Any thoughts on this, particularly issues I haven't thought of?

ChucuSCAD · Post by **ChucuSCAD** » Mon Feb 07, 2005 3:59 pm

Well you should be able to do ok with that setup. The key is getting good gigabit cards and good hard drive controllers. Something in the higher end 3com area for your network and switches. Controllers probably 3ware if you had some bucks... but adaptec or the like will do just fine.

Cheaping out in either area will effect speed big time. Yes a faster cpu/board will help out but not really. Many businesses still have file servers on older P2/P3 native and Xeon based systems. The one other thing you can do is max out the ram.

chucuSCAD

SometimesWarrior · Post by **SometimesWarrior** » Mon Feb 07, 2005 4:24 pm

Take a look at this Ace's Hardware article where they compare nForce3 motherboards, and you will see what happens when you use a cheap GigE adapter. The onboard adapters in the article can get close to maxing out a top-of-the-line Athlon 64 when transferring data at high speed!

I think a high-quality adapter will offload most of the work from the processor, but perhaps not enough for your system. Though I have no experience with GigE, my guess is that the processor will still be the limiting factor even if you switch to the higher-speed K6. Whether it will limit the transfers to 100Mb/s or 600Mb/s I can't even pretend to know.

How much of the hardware do you have now? Do you have gigabit cards, or a gigabit switch? I suggest running some benchmarks before making any big purchases. Ace's Hardware uses the PassMark benchmark program, which has a free 30-day trial. If you have the K6 machine hooked up now, try the benchmark on fast ethernet, and see what kind of CPU load you get. If you have the gigabit-capable cards now, but no switch, try using a crossover cable to perform a test. If network speeds aren't up to snuff, then it might be cheaper to get a faster processor and slower ethernet card than it is to get a top-of-the-line ethernet card.

I wonder how the CPU, network adapter, and fast/gigabit all affect network response time? Are the hard drives still the primary source of latency, or will a network drive be much more sluggish for things like application-serving, not considering the possibly higher CPU usage? For only storing sequential files like music and downloads, latency won't be a big deal, but it could limit the usefulness of a network file server.

taqueso · Post by **taqueso** » Mon Feb 07, 2005 4:24 pm

You may find this page interesting: http://www.syskonnect.com/syskonnect/pe ... copper.htm

Beware that the study is performed by the makers of the "winning" card.

I ordered my first GigE equipment yesterday, so I can't really comment intelligently on any aspects of it. FYI, I bought Intel Pro 1000 MT cards because the linux drivers support all of the card features (jumbo frames, interrupt combining).

jack_aubrey · Post by **jack_aubrey** » Mon Feb 07, 2005 4:36 pm

Thanks all, this is great feedback. Re the hardware, all I need are the GigE adapters, switch, and some Cat6 cable. The comments about the components are well taken, I'd think interrupt coalescence is particularly important with a (relatively) slower CPU like this.

This will be primarily a Linux client network, and a Linux server, so PassMark may not be that apropos as I'll be using a completely different software and driver stack. But there are some OSS I/O benchmarks that may provide comparable information. I'll mostly be serving audio and video over NFS (and maybe Samba/CIFS), also some software builds; I'm hoping to be able to read and burn DVDs on a client, without dramatic performance loss compared to client-local storage.

taqueso, good pointer on the Intel cards - when you were researching those, did you come across any other promising alternatives? I'd be interested in hearing how well your setup comes out.

burcakb · Post by **burcakb** » Mon Feb 07, 2005 11:07 pm

If you're not simultaneously serving several clients, I doubt you'll need GigE.

I watch divx movies fine over 54g wireless.

jack_aubrey · Post by **jack_aubrey** » Mon Feb 07, 2005 11:13 pm

burcakb wrote:If you're not simultaneously serving several clients, I doubt you'll need GigE.

Maybe not, but it is much a hobby project as anything else. Need something to putter with on the evenings I'm not out dancing.