Poor Man's iRAM with USB thumbdrives?

IsaacKuo · Post by **IsaacKuo** » Tue Apr 25, 2006 10:58 am

Is it feasible to get relatively decent speeds from a USB thumbdrive RAID0? I'm thinking of RAIDing three of these:

RiDATA MiniSPIN 512MB Flash Drive - $10

Even with shipping, three of them would only cost $45, and would total 1.5gigs of space--plenty of space for a lean Debian install. It's cheap enough that I really don't care about risking the flash rewrite limit.

Based on the NewEgg customer reviews, I estimate 9+ megs/sec read speed. RAID0 should boost it up to 27 megs/sec, right? Or are there USB2.0 limitations which will cap performance to something lower?

Is this idea just totally stupid?

andrewln · Post by **andrewln** » Tue Apr 25, 2006 11:55 am

new hard drives can do at least 60Mb per second

Iram is good for seek time,roughly around 0.02ms....hard driveare 4+ms if you get raptor.

USB flash drives has limited write cycles...like 10,000 times...thats it

As far as i can see...there is no real benifit for you to do this

IsaacKuo · Post by **IsaacKuo** » Tue Apr 25, 2006 12:08 pm

Hard drives make noise!

And while my cheap homebrew enclosures can effectively silence a quiet notebook drive, the cost of the enclosure plus 2.5"->3.5" adapter plus the notebook drive can easily be twice that of this thumbdrive solution.

Devonavar · Post by **Devonavar** » Tue Apr 25, 2006 3:01 pm

I doubt you will see your throughput triple with RAID 0. IIRC, the benefit is typically much less than the total bandwidth combined. Why not poke around in Storagereview and see what they say?

@andrewln: I believe you are confusing disc latency, seek time, and access time.

Because the iRAM uses solid state storage, there is no seek time or disc latency, just a short transfer delay. This is comparable to access time on a hard drive, not seek time. Average access time is equal to the average latency (~3 ms in the case of the 150 GB Raptor) plus the average seek time (4.6 ms), for a total of 7.6 ms.

Copper · Post by **Copper** » Tue Apr 25, 2006 5:01 pm

Is there even any kind of hardware that would let you get USB drives into RAID?

Post by **NeilBlanchard** » Tue Apr 25, 2006 5:18 pm

Flash drives in RAID 0 array

IsaacKuo · Post by **IsaacKuo** » Tue Apr 25, 2006 5:40 pm

Thanks for the link! I had found the first Arstechnica test, but like that second one they were only thinking in terms of a portable drive (which is a really stupid idea due to the impracticality of getting it to work on any other computer).

They obviously hadn't even considered the possibility of using them as the main mass storage of a computer.

The performance results seem at least somewhat encouraging, although it's complicated by lack of information on stripe size and I can't help but wonder what effect the "journaling" file system has. I'm not familiar with Mac OS/BSD file systems; I know in Linux the ext3 journaling file system constantly periodically writes to the drive so I'd never use it for a flash device. I'll be using linux's ext2 file system.

Well...I'm getting one of these flash drives from NewEgg for a car MP3 player regardless. I just want to confirm the size to see if I can cram two of them together in adjacent USB slots. If so, then I'll give this idea a try. I've got four USB slots; saving one slot for removeable devices that leaves me with three.

Is there even any kind of hardware that would let you get USB drives into RAID?

I'll be using linux's software RAID.

Copper · Post by **Copper** » Tue Apr 25, 2006 6:06 pm

IsaacKuo wrote:I'll be using linux's software RAID.

That's pretty cool. At $15 a whack, why not. It'd be fun to play with regardless.

mb2 · Post by **mb2** » Sat May 20, 2006 1:48 pm

this is a very interesting project. i think the fact that the access time is so low, may help raid0 performance compared to normal hdds?.
dont forget that usb drives also have very low access times (not *as* low, but much lower than hdd), increased i imagine by the usb bus (?). but its the sustained transfer that also is great in the iRAM.
M$ is using them to increase performance with vista, so i think thats enough of a sign that in 'OS' conditions they increase responsiveness.(?)

anyway, any progress?
i would suggest looking at microwinX, but since ur gonna use linux, thats pretty moot. and afaik there is no way u could use raid with windows (?).
i like the idea of having one thou, then u can take with with u and boot ur os on anyones PC (that supports usb boot obviously..). dont think the whole activation stuff will like that lol.

IsaacKuo · Post by **IsaacKuo** » Sat May 20, 2006 3:46 pm

I have been struggling to get it to work, using Debian Etch. It's very frustrating because I feel like I've been getting really close to it working.

I've been taking a break from this project out of frustration...I'll get back to it, though.

bexx · Post by **bexx** » Mon May 22, 2006 11:33 am

I've been planning on using linux md raid on compact flash cards with cf->ide converters. 2GB card is ~$50 for a 100x card or so.. I havn't really decided on the exact card yet.

My plan is 4 of these cards basically, with card0 i'll have a 256MB boot partition (yea sorta a waste), then card1,2,3 will have 256MB partitions that are raid0 and it will be an encrypted drive for basically documents and stuff.... the other 1750MB of each card will be put into raid0 for a root drive giving ~7GB of space.

I'll have a NFS file server for alot of bulk storage and try to offload as much of the os as I can onto it... including /tmp and /home.

Everyone says this is a horrible idea so I've been putting other projects ahead of it... next couple of pay checks though I'm just going to order it and see what happens.

It should work, I mean the cf cards act exactly like an ide hard drive, reliabilty is the only real concern.

IsaacKuo · Post by **IsaacKuo** » Mon May 22, 2006 12:00 pm

My original plan was also based on CF cards, but unfortunately prices on CF cards still haven't gotten low enough for me (and maybe they never will). At current prices, I'd rather just go with iRAM and get much higher performance.

Anyway, if you do use RAID0 with IDE drives, be aware that you should NOT RAID0 together more than one partition per IDE channel. Most motherboards have only two IDE channels, so that means raiding together two drives.

bexx · Post by **bexx** » Mon May 22, 2006 10:12 pm

Well price was 1GB dimms are down to maybe $80 now? Just for 8GB of ram it'll be $640, but tehn you need 2 iram controllers still. 2GB dimms would help maybe but you end up paying more per MB which offsets the savings of using 1 controller. Although I am kinda annoyed with the cost of CF when you compare it to SD cards. I'm sorta settling with average quality brands to get 2GB cf cards @ $50 because a nice sandisk is $100+ still, yet 2GB SanDisk SD cards are $50.

8GB of flash = $50*4+2*8=$216.

Its not as cheap as usb sticks however I've always gotten horrible performance to my keychain usb stick and generally hate dealing with usb much less having to boot off it.

As for 2 drives per cable, its not ideal but I'm hoping for 10MB/s per card... I won't be running into any bandwidth limitations, Add-in pci ide controllers are like $10-$15 so worst case I grab one of those.

IsaacKuo · Post by **IsaacKuo** » Mon May 22, 2006 10:59 pm

bexx wrote:As for 2 drives per cable, its not ideal but I'm hoping for 10MB/s per card... I won't be running into any bandwidth limitations, Add-in pci ide controllers are like $10-$15 so worst case I grab one of those.

Due to the way IDE works, you can't get more bandwidth with more than one drive per cable. Only one request can be handled at a time and the entire channel is locked until that request is completed. This contrasts with USB or SCSI, where multiple requests can be handled at the same time.

bexx · Post by **bexx** » Tue May 23, 2006 12:16 am

http://www.xbitlabs.com/articles/storag ... md680.html

Theres basically teh controller I'd be using and they compare 2 channel arrays to one channel arrays. Often 1 channel is similar to 2 channels, sometimes it gets beat out bad but most of those tests show small differences. 4drive raid0 is always faster than 2 drive raid0 I don't think one test it was worse. I dunno I'll see how it works. Its more just a for-fun project

IsaacKuo · Post by **IsaacKuo** » Tue May 23, 2006 4:38 am

Ah--I see. Yes, write performance can be good because there's no need for feedback for a write request. However, read performance is IMHO more important because that's what's done most of the time.

Note that for reads, two drive two channel performs twice as well as two drive single channel; four drive hardly performs any better than two channel (and this could easily be simply because of being able to store data in the faster outer tracks--an effect which doesn't apply to solid state drives).

highlandsun · Post by **highlandsun** » Thu Jul 20, 2006 12:37 pm

40MB/sec 8GB CF cards from SanDisk - yum...
http://www.dailytech.com/article.aspx?newsid=3419

IsaacKuo · Post by **IsaacKuo** » Thu Jul 20, 2006 12:46 pm

Wow, those are awesome! But at those prices, I'd sooner get an iRAM.

I've given up on RAIDing multiple USB thumbdrives--I just can't get it to work. I think someone who really knew what he was doing could get it working, but it's beyond my expertise.

Right now, I've scaled back my ambitions to using just one USB thumbdrive. The performance will be poor, but at least it'll work. I've been working on various other computer upgrades, and actually just today started back on this project.

IsaacKuo · Post by **IsaacKuo** » Fri Jul 28, 2006 6:12 am

Ah, sweet success! No, I didn't succeed in booting from a thumbdrive RAID. Instead, I finally decided to figure out how to do diskless net-boot. I'm shocked--the performance even with mere 100mbit ethernet is very good.

Since I'm not using gigabit, I thought it would be sluggish. In fact it feels as responsive as using a local drive. In particular, I'm highly impressed by the boot-up time and the time to startup Firefox.

I guess my new "Poor Man's iRAM" is to diskless netboot, with a ton of RAM installed (which acts as a fast disk cache).

Assuming I have time for it, I'll be writing up my diskless net-boot project this weekend.

highlandsun · Post by **highlandsun** » Tue Sep 05, 2006 11:24 am

Newegg has a 100X 8GB CF card for $129.99 now. It's getting to be a tempting option for a laptop. These guys http://mesanet.com/ have a CFADPT7-D CF-IDE adapter that mounts in a 2.5" HDD mount, with dual CF slots, for $25. I could get a lot of use out of a Linux laptop with 16GB of disk, and just keep upgrading as higher density CF cards come out.

32GB (2 IDE channels, 4 CF cards) may be a bit small for a silent HTPC, not sure if keeping main storage in external firewire drives (320GB WD3200s most likely) would help or hurt.