Brainstorm with Me About Storage and Computing Solutions
Posted: Mon Nov 06, 2006 3:15 pm
I would like some help brainstorming for some possible solutions to my computing needs. I have about 900 GB spread over 9 IDE hard drives which are all essentially full. My case is also at its limit in the number of drives that can be stuffed into it. Some of the files on the drives are duplicated or burned to DVD but the vast majority are not and it is a disaster waiting to happen. The price of hard disk storage space is approaching that of DVDs especially if I factor in their ease of use and account for my time. I would like your help in coming up with ideas of how I could create redundant relatively safe storage on a cheap budget (I am a grad student). What ever I end up using I would like it to be relatively expandable and quiet since I live in a small apartment.
Overall Needs Ranked in Order of Importance (Further Items Might Not Happen Any Time Soon):
1) Storage Solution:
3) Another machine to mount another BenQ 1655 so that as I am burning DVDs I can be doing PI/PO scans using Nero CDSpeed in parallel
4) Improved interfaces:
6) New machine all to run my numerically and disk intensive simulations at home (FEA).
I have the following hardware currently available:
Desktop (Windows XP SP2) (This is the main machine that I use at home day to day):
AMD Athlon 1.33 GHz (CNPS7000B-AlCU CPU Cooler)
512 MB DDR 266 RAM
IWill KA266+ (USB ports do not function I think because I accidently cut some traces on the MB) (2 PCI slots left)
ANTEC SLK3000B Case (Yate Loon 120 mm fan mounted to cool 5 hard drives)
ANTEC Truepower 2.0 430 W PSU
Matrox Millenium G550
3Com Etherlink XL 10/100 PCI Ethernet Card
Promise Ultra100 Tx2 IDE Controller Card
Promise Ultra133 Tx2 IDE Controller Card
BenQ 1655 DVD Burner
NEC 3540 DVD Burner
Kingwin Mobile Rack
CoolerMaster 4 in 3 Hard Drive Mount (This is mounted in the top three 5.25 bays)
Laptop (Windows XP SP2):
Dell 4100 (600 Mz P3)
40 GB Samsung Laptop Hard Drive
Network Router:
Linksys WRT54G v4
I am leaning to some sort of RAID6 solution because of problems RAID5 has with rebuilds and uncorrectable read errors. I relize that RAID is not a backup but I have not seen a viable backup solution for ~1 TB at a cost I am willing to pay. As far as I can tell only recent PCIe or PCI-X hardware raid controllers support RAID 6 expansion. Except for the problems with growing the array the md linux solution looks to be the most cost effective unfortunately I would like something that I can expand to account for future storage needs.
A) Are there any other solutions that I should consider or gotchas that I haven't listed below.
B) Which way would you go and why?
C) If I go the Linux route and pick up an older motherboard and processor what would you recommend? What current heatsinks and fans would work with it? Is there a specific used components that I could ebay that would be relatively quiet?
Ideas that I am considering for storage solutions:
Solaris x86 using ZFS
Pros:
Linux Software RAID5/6 on an Old Machine
I could pick up a cheap P3 motherboard, CPU, RAM, Case, and PSU, and some PCI sata cards and make a linux software raid with md and maybe EVMS.
Pros:
Pros:
Two recent hardware RAID cards support RAID 6:
Areca ARC-1220 PCI-Express x8 Sata II Controller Card - $494.00
3ware 9590SE-8ML PCI Express x4 SATA II - $504.00
Pros:
Pros:
B) Which way would you go and why?
C) If I go the Linux route and pick up an older motherboard and processor what would you recommend? What current heatsinks and fans would work with it? Is there a specific used components that I could ebay that would be relatively quiet?
Overall Needs Ranked in Order of Importance (Further Items Might Not Happen Any Time Soon):
1) Storage Solution:
- ~1 to 1.5 TB of redundant hard drive storage
- Relatively cheap beyond the costs of the hard drives themselves
- Expandable
- Quiet
- Easy managment (Preferably would appear as a single volume)
- Low Power
- Directly accessible from Windows (Samba share would probably work though I would prefer it to be directly connected)
3) Another machine to mount another BenQ 1655 so that as I am burning DVDs I can be doing PI/PO scans using Nero CDSpeed in parallel
4) Improved interfaces:
- USB 2.0 - Neither of my computers has USB 2.0 I would like to get an IPod Nano soon so USB 2.0 connections would be useful.
- Sata II - I would prefer not to buy anymore IDE drives as they are a dying technology and the cabling restricts airflow so much.
6) New machine all to run my numerically and disk intensive simulations at home (FEA).
I have the following hardware currently available:
Desktop (Windows XP SP2) (This is the main machine that I use at home day to day):
AMD Athlon 1.33 GHz (CNPS7000B-AlCU CPU Cooler)
512 MB DDR 266 RAM
IWill KA266+ (USB ports do not function I think because I accidently cut some traces on the MB) (2 PCI slots left)
ANTEC SLK3000B Case (Yate Loon 120 mm fan mounted to cool 5 hard drives)
ANTEC Truepower 2.0 430 W PSU
Matrox Millenium G550
3Com Etherlink XL 10/100 PCI Ethernet Card
Promise Ultra100 Tx2 IDE Controller Card
Promise Ultra133 Tx2 IDE Controller Card
BenQ 1655 DVD Burner
NEC 3540 DVD Burner
Kingwin Mobile Rack
CoolerMaster 4 in 3 Hard Drive Mount (This is mounted in the top three 5.25 bays)
Laptop (Windows XP SP2):
Dell 4100 (600 Mz P3)
40 GB Samsung Laptop Hard Drive
Network Router:
Linksys WRT54G v4
I am leaning to some sort of RAID6 solution because of problems RAID5 has with rebuilds and uncorrectable read errors. I relize that RAID is not a backup but I have not seen a viable backup solution for ~1 TB at a cost I am willing to pay. As far as I can tell only recent PCIe or PCI-X hardware raid controllers support RAID 6 expansion. Except for the problems with growing the array the md linux solution looks to be the most cost effective unfortunately I would like something that I can expand to account for future storage needs.
A) Are there any other solutions that I should consider or gotchas that I haven't listed below.
B) Which way would you go and why?
C) If I go the Linux route and pick up an older motherboard and processor what would you recommend? What current heatsinks and fans would work with it? Is there a specific used components that I could ebay that would be relatively quiet?
Ideas that I am considering for storage solutions:
Solaris x86 using ZFS
Pros:
- ZFS management sees relatively easy and well documented
- Checksumming would help data corruption problems
- RAIDz2 is essentially equivalent to RAID 6
- Limited hardware compatibility (It is possible I would need a new MB, CPU, and RAM)
- Without a 64 Bit CPU the amount of memory that ZFS can use is limited to 512 MB and apparently has some impact on performance.
- It is not clear if Samba works correctly? (It seems to be built in but there are problems with ACLs)
- It is not currently possible to add devices to a raidz vdev. I couldn't expand the vdev by adding the same size disks.
- Solaris seems to be lacking a good package managment system. Nexata looks interesting but it is still in beta and doesn't update the core OS.
- I am not familiar with Solaris administration and updating.
Linux Software RAID5/6 on an Old Machine
I could pick up a cheap P3 motherboard, CPU, RAM, Case, and PSU, and some PCI sata cards and make a linux software raid with md and maybe EVMS.
Pros:
- Potenially cheap hardware except for sata cards, case, and psu.
- RAID 6 arrays cannot be grown currently
- I have not used Linux/Unix in a serious way for at least 8 years
- No vendor support for RAID solutions
- I have enough old hardware already
- Unclear what type of bit scrubbing md does to avoid uncorrectable read errors
Pros:
- Easy setup and administration
- X-RAID allows expansion as I can afford disks
- Low power
- Limited to 4 disks (I wish this had 8 bays)
- Cost (~$650 for the device without disks)
- Less flexible than a general linux installation
- RAID 6 is not available
- Unclear what type of bit scrubbing the device does to avoid uncorrectable read errors
Two recent hardware RAID cards support RAID 6:
Areca ARC-1220 PCI-Express x8 Sata II Controller Card - $494.00
3ware 9590SE-8ML PCI Express x4 SATA II - $504.00
Pros:
- I could keep my storage directly accessible in a Windows machine
- Easy administration
- Hardware compatibility lists for motherboards and hard drives
- Battery backups add at least another $100 dollars
- Cost and I would need to purchase a new MB, CPU, and RAM.
- The hardware controller becomes a single point of failure.
- Not as easily sharable to other computers in the future.
Pros:
- Relatively inexpensive
- Potenially directly attached to my current machine
- Loss of half my storage space to mirror drives.
- PCI bus in my current machine is already fairly crowded.
B) Which way would you go and why?
C) If I go the Linux route and pick up an older motherboard and processor what would you recommend? What current heatsinks and fans would work with it? Is there a specific used components that I could ebay that would be relatively quiet?