Page File to Different Drive + RAID ?'s

Silencing hard drives, optical drives and other storage devices

Moderators: NeilBlanchard, Ralf Hutter, sthayashi, Lawrence Lee

Post Reply
Sainty
Posts: 55
Joined: Tue Aug 21, 2007 7:48 am

Page File to Different Drive + RAID ?'s

Post by Sainty » Fri Aug 24, 2007 8:55 pm

Before we get started let me explain the hardware I have and what I intend to use it for:

Two (2) Deskstar 82.3gb 7200RPM SATA 1.5 hardrives, (Previously used in a RAID 0 setup under Windows XP Pro)
One (1) 160gb Seagate 7200RPM IDE hardrive.

My mobo is this guy:

http://www.gigabyte.com.tw/Support/Moth ... uctID=2533

The mobo supports Raid 0,1,5 and 10.

I was thinking about putting the Deskstars in RAID 0 and just using the Seagate as file storage, with the RAID array handling the OS.

However, after reading the following article I'm wondering if RAID 0 isn't the way to go, considering the increased risk of failure and marginal performance gains:

http://www.anandtech.com/storage/showdoc.aspx?i=2101

See also:

http://www.anandtech.com/storage/showdoc.aspx?i=2974

RAID 1 looks more like the way to go in terms of nominal performance differences with the reliability to boot. The issues comes down to storage space. If I ran a RAID 1 with these two drives it will only leave a small amount of space to store additional non-OS files. Is it important that the game files be installed on the RAID array for the best performance compared to my single IDE drive?

Switching gears slightly, I've also heard of people using a separate hardrive for their paging file to increase performance. Should I still run a RAID 1 setup on

Here's some possible configurations:

1)

RAID 1 (put OS on the array and installed games, keep array as standard page file)
IDE hardrive for additional storage only

2)

RAID 1 (put OS on the array and installed games)
IDE hardrive for additional storage and the page file

3)

Suggest a better setup?

I'm at a bit of a loss as to what to do. I'm just about broke, so purchasing extra hardrives isn't an option at this time. They keep on increasing in size and coming down in price so this could be a future upgrade. However, in the mean time I really need to get something up and running soon. Given the hardware and usage patterns outlined above, what's the best configuration?

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Fri Aug 24, 2007 11:52 pm

Get 2GB of RAM and turn the page file off completely.

I have done just that and can run all games etc without problems, even things like Supreme Commander and Dirt. Most XP machines only have 2GB VM total anyway, at least with the default settings.

Sainty
Posts: 55
Joined: Tue Aug 21, 2007 7:48 am

Post by Sainty » Sat Aug 25, 2007 12:02 am

The machine that is being built will have 4gigs of RAM in a 2x2gb configuration. The OS will be Vista 64bit.

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Sat Aug 25, 2007 1:00 am

In that case you certainly don't need a page file.

Mobo based RAID controllers are usually rubbish. They are usually more trouble than they are worth too - I used to use a Promise on-board RAID and every now and then it would "forget" one drive was part of the array. Recreating the array worked without data loss, but I got fed up of buying new underwear.

If you want a significant performance boost, either get a proper raid controller like an Areca or maybe an SSD for the OS. Both are expensive, but... Oh, and Areca RAID controllers support SMART and can report drive temps, which cheaper controllers cannot.

nick705
Posts: 1162
Joined: Tue Mar 23, 2004 3:26 pm
Location: UK

Re: Page File to Different Drive + RAID ?'s

Post by nick705 » Sat Aug 25, 2007 1:25 am

Sainty wrote: I'm at a bit of a loss as to what to do. I'm just about broke, so purchasing extra hardrives isn't an option at this time. They keep on increasing in size and coming down in price so this could be a future upgrade. However, in the mean time I really need to get something up and running soon. Given the hardware and usage patterns outlined above, what's the best configuration?
Given your circumstances, I'd go with the first option - use the two Deskstars in RAID-0 for the OS/apps, and the Seagate for your important data.

I wouldn't normally have any use for RAID-0, but your Deskstars are quite small and slow by today's standards, and striping *may* be the best way to squeeze the most out of them. The reliability factor isn't so much of an issue with an OS volume, unless downtime is important to you.

*Slightly OT: I think of all the arguments against RAID-0 (and I agree with most), the "reliability" issue is somewhat overblown. To hear what some people say, you'd think any data placed on a striped volume was in constant, imminent danger of catastrophic, irretrievable loss - admittedly, the risk is doubled compared with a single drive over a given period, but in reality the risk has increased from "small but not negligible" to "still small, but somewhat larger and not negligible." The real point is that *any* hard drive, whether in isolation or as part of an array, is subject to failure at any time, and if the data on it is of any value, you'd be silly not to have it backed up. If a single drive fails, and you don't have a backup, it's not much comfort knowing the risk was half of what it would have been in a RAID-0 array...
Sainty wrote:The machine that is being built will have 4gigs of RAM in a 2x2gb configuration. The OS will be Vista 64bit.
With so much physical memory, I think the location and size of the pagefile will be mostly irrelevent. If you want, you can put a pagefile on each drive (the OS array and the Seagate data drive), and Windows will automatically use the one that's least busy. :)

AZBrandon
Friend of SPCR
Posts: 867
Joined: Sun Mar 21, 2004 5:47 pm
Location: Phoenix, AZ

Post by AZBrandon » Sun Aug 26, 2007 11:03 am

You need to have a really, really good reason to run a raid-0 array. I used to have 4 drives in my P180 in raid 10 (0+1) using the onboard NVIDIA raid controller. It worked well enough until it decided to not boot up anymore. Fortunately, I do backups all the time with Acronis True Image, so rebuilding wasn't a big deal, but I still lost several days worth of data since I only do backups weekly.

Since I really had no legitimate reason to need the added throughput you get with raid 10 I just disabled it and took all the drives out but one and began doing my weekly backups to an eSATA drives, rotating two different physical drives for redundancy and the ability to physically separate the backups from the PC.

Considering any modern SATA can do above 50mb/sec of sustained streaming throughput for stuff like HD video capture or whatever else you may have conjured up for your data, there's little chance any of your apps really need more than that. If you did, I would probably suggest getting serious and buying a caching raid controller with its own onboard memory. As for paging, I have "only" 2gb of ram and haven't had a pagefile in as long as I can remember. The only time I've ever gotten an out of memory error was when I left a ton of applications running and loaded up Battlefield 2.

lm
Friend of SPCR
Posts: 1251
Joined: Wed Dec 17, 2003 6:14 am
Location: Finland

Re: Page File to Different Drive + RAID ?'s

Post by lm » Sun Aug 26, 2007 2:00 pm

Sainty wrote: Two (2) Deskstar 82.3gb 7200RPM SATA 1.5 hardrives, (Previously used in a RAID 0 setup under Windows XP Pro)
One (1) 160gb Seagate 7200RPM IDE hardrive.
I would buy a single 320GB or larger SATA2 drive, and sell all three old drives.

- single dual platter drive is much quieter than three drives.
- much faster than the old ones
- less wasted power is less heat -> again less noise
- less cable clutter, especially losing the IDE, ugh
- increased reliability by ditching the old drives

Cerb
Posts: 391
Joined: Tue Apr 13, 2004 6:36 pm
Location: GA (US)

Post by Cerb » Sun Aug 26, 2007 2:17 pm

2GB: 2GB is the VM limit per app. If you don't multitask much, or leave your apps running, then yes, it effectively becomes a real physical limit as far as how much you use. I don't see how anyone manages that way, but then again, I have FF up all the time, with 100-200 tabs, and that alone will use 1GB or more. A page file is not bad--Windows just isn't very good at managing one (Vista does a decent job at band-aiding the problem, but it's still not great).

RAID: for most users, backup with an app like Acronis' is the way t go. Do daily backups, come on! Middle of the night is good; or maybe have it do a backup when you leave for work in the morning (that way, while manual, you can still take advantage of S3). You could also mess with WoL to get things done, too. The problem with RAID is that if something corrupts a file, that's propagated. You're only protected from a catastrophic drive failure.

Also, such backups would allow you to use any RAID scheme, too (lke RAID 0 on the deskstars and backups to the Seagate).

If you want a nice little real RAID controller, pass over Areca, and get a 3Ware. Arecas are very expensive, and have small fans. 3Ware report SMART stuff just fine, and work well, though their performance isn't earth-shattering.

Personally, I use a single drive, and manually back up my data. Less parts, less noise, less fuss...but more risk, and greater downtime.

Page file: a separate drive for the page file from your other stuff can help, but not that much. Its main benefit is if you're using the HDD a lot and needing to page in/out. If you edit big videos, maybe that is important, but otherwise, it's not going to help enough to be worth doing.
Last edited by Cerb on Sun Aug 26, 2007 3:30 pm, edited 1 time in total.

nick705
Posts: 1162
Joined: Tue Mar 23, 2004 3:26 pm
Location: UK

Re: Page File to Different Drive + RAID ?'s

Post by nick705 » Sun Aug 26, 2007 2:33 pm

lm wrote: I would buy a single 320GB or larger SATA2 drive, and sell all three old drives.

- single dual platter drive is much quieter than three drives.
- much faster than the old ones
- less wasted power is less heat -> again less noise
- less cable clutter, especially losing the IDE, ugh
- increased reliability by ditching the old drives
I guess you missed the bit where the OP said: "I'm just about broke, so purchasing extra hardrives isn't an option at this time."

Lots of problems can be solved by throwing money at them, trying to make the best of limited resources is a bit more challenging.

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Sun Aug 26, 2007 7:08 pm

Cerb wrote:2GB: 2GB is the VM limit per app. If you don't multitask much, or leave your apps running, then yes, it effectively becomes a real physical limit as far as how much you use. I don't see how anyone manages that way, but then again, I have FF up all the time, with 100-200 tabs, and that alone will use 1GB or more. A page file is not bad--Windows just isn't very good at managing one (Vista does a decent job at band-aiding the problem, but it's still not great).
100-200 tabs!? I sometimes get up to maybe 30-40 (rarely), using maybe 500MB of RAM. I tend to keep FF, The Bat and various other apps open all the time, but have never had an issue with running out of memory.

Still, with 4GB RAM... look at it this way. A default XP install on 512MB RAM gives you a max 1.5GB page file, so 2GB VM total. Due to address space being reserved for PCI devices etc, most 32bit systems can only have 3.5GB addressable memory (i.e. total VM, paged and physical) anyway. If you run out of memory with 4GB physical RAM on a 32 bit system, you are screwed anyway and the only solution is to move to a 64 bit system.

Unless you like running FF with 200 tabs, Supreme Commander and a few Prime95 blend tests at the same time I can't see it being much of a problem :) Try it and see is my advice, Windows will warn you if you run low on memory and you can re-enable the page file any time.
Cerb wrote:RAID: for most users, backup with an app like Acronis' is the way t go. Do daily backups, come on!
Good advice. There is a free Microsoft tool called SyncToy which is quite handy for quick, scheduled backups if you don't want to do a full drive image.

One thing to note about all drive imaging software though: I have yet to find anything that works with dynamic drives in Windows. Basic drives are fine, but if you convert them to dynamic you will be able to image partitions but not restore them. Restoring works if you format the drive back to basic and restore to a basic partition, but that of course means you have to wipe all partitions on the drive. Not much use if you want to roll back to before a bad update or virus infection etc.
Cerb wrote:If you want a nice little real RAID controller, pass over Areca, and get a 3Ware. Arecas are very expensive, and have small fans. 3Ware report SMART stuff just fine, and work well, though their performance isn't earth-shattering.
All Areca cards come with a fanless heatsink in the box that you can fit without voiding the warranty. As long as your case has reasonable air flow, you can use the fanless heatsink.

Lensman
Posts: 147
Joined: Thu Jul 26, 2007 1:15 am

Post by Lensman » Sun Aug 26, 2007 8:42 pm

So is it really common knowledge that Intel's ICH9R Matrix Raid controller is rubbish? I'm kind of asking for myself as well since I was going to use it.

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Mon Aug 27, 2007 1:12 am

Well, newer motherboard based RAID controllers are a bit better in terms for reliability and even performance, but generally they are still a pain in the arse.

Most on-board controllers only have a basic BIOS and no processor (so really they are software RAID controllers with a tiny bit of hardware assist). Performance is often fairly rubbish as well, again due to basically being software RAID and not having any cache memory etc.

The only real advantage over Windows software RAID is that you can use drive image tools to back the RAIDed drives up from DOS.

I gave up with on-board RAID and now just use plain SATA drives. No drivers required in an emergency, easy to connect the drives to another PC if the worst happens. Keep that in mind - if your mobo dies for some reason you will need another identical RAID controller to access the data. Try buying one of those in a year or two.

In actual fact, sometimes non-RAID performance is better than RAID performance. Cheaper on-board RAID controllers can impact seek times, and if you are copying disk to disk (say when extracting a video file you downloaded from a temp drive to a storage drive) it is actually often faster than a RAID 0 array doing both reads and writes.

Cerb
Posts: 391
Joined: Tue Apr 13, 2004 6:36 pm
Location: GA (US)

Post by Cerb » Mon Aug 27, 2007 11:37 am

Didn't know Arecas came with another HS. Interesting.

2GB VM: 2GB VM is the limit in 32-bit regardless of how much RAM you have. It's a WIndows/IA32 limit. Technically, it can be gotten around with PAE and /3GB, but both cases need the program to be written for it.

If you've got 128MB RAM, you're still limited to 2GB/app (2GB/app, 2GB kernel, but very little kernel space tends to be used), just as if you've got 8GB RAM. It has nothing to do with your actual amount of RAM, except that with modern hardware, the limit can be reached on a home PC. For someone not keeping track of many tasks, that's one reason, along with video eating up the top end in nice gaming boxes, not to go over 2GB physical.

The OP going w/ Vista64, of course, makes it mostly moot, though.

FF tabs: Tab Mix Plus allows multiple rows of tabs, so it's really easy, especially with session saving.

Arvo
Posts: 294
Joined: Sat Jun 10, 2006 1:30 pm
Location: Estonia, EU :)
Contact:

Post by Arvo » Mon Aug 27, 2007 12:14 pm

Well, I'd avoid any RAID in your case. To maximize performance I'd use disks as follows:

(1) 80GB#1: Boot, OS, ProgramFiles + storage (once written, often read files)
(2) 80GB#2: 2GB page file, fixed size, at start of disk + storage (once written, never or rarely read files)
(3) 160GB partition 1, about 40-80GB: Volatile data (user profiles, including temp folders), some other programs working folders etc
(4) 160GB partition 2, remaining space: storage (average r/w frequency)

This way swapping on (2) (if occurs) does almost not interfere with other disk activity and is very quick - no long seeks. System and ProgramFiles disk (1) is mostly read and will be nicely cached, most random+write activity will happen on first part of fastest disk (3).

Of course this schema is based on my experience and disk usage pattern, yours may be totally different.
For example, if you need some almost uncompressed video stream recording space, then this doesn't fit at all (I've another, usually empty HDD for this purpose).

Nick Geraedts
SPCR Reviewer
Posts: 561
Joined: Tue May 30, 2006 8:22 pm
Location: Vancouver, BC

Post by Nick Geraedts » Mon Aug 27, 2007 3:05 pm

MoJo wrote:Get 2GB of RAM and turn the page file off completely.

I have done just that and can run all games etc without problems, even things like Supreme Commander and Dirt. Most XP machines only have 2GB VM total anyway, at least with the default settings.
Hang on... Hang on. This is a strange misconception about Virtual Memory.

With 32-bit systems, the maximum amount of address space there is in total is 4GB (2^32 bits). Applications have a virtual address space of 2GB each. For 64-bit systems, the rules are completely re-written, so you basically won't run out of address space in this day and age.

What's the problem with having a pagefile anyways? Windows will only use it when necessary (even XP), and don't get me started on the PF Usage graph in Task Manager - that's not pagefile usage!


Regarding onboard RAID controllers - for RAID0 and RAID1, onboard will do just fine. There are no extra calculations required, so you won't notice any performance boost from those setups to a dedicated controller. The ICH7R and up are all fine for RAID0,1, and 10 setups. If you want to go to a heavy usage RAID5 setup, you'll want to look at a hardware RAID solution. The setup for the Intel Matrix controller is also very easy... a monkey could use it!

jaganath
Posts: 5085
Joined: Tue Sep 20, 2005 6:55 am
Location: UK

Post by jaganath » Mon Aug 27, 2007 3:11 pm

What's the problem with having a pagefile anyways? Windows will only use it when necessary (even XP),
AIUI, a pagefile is an area on the hard disk that Windows treats as if it were RAM. in my case, I have a 5400rpm system drive, so it is the slowest part of my system. it does not seem very sensible to store information that is needed quickly on the slowest part of my system, hence no pagefile. also, the amount of HDD seek noise that is created by it is very annoying.

andyb
Patron of SPCR
Posts: 3307
Joined: Wed Dec 15, 2004 12:00 pm
Location: Essex, England

Post by andyb » Mon Aug 27, 2007 3:32 pm

I removed my Page File from my Desktop on the basis that it will reduce the hard drive seeks and make my PC even quieter, it worked, and along with disabling HDD caching for Firefox I was very happy.

As it turned out I found a performance improvement with no page file and 2GB or RAM that I have been using for a couple of years, I wont be going back.

RE: RAID setups.
If you want to go to a heavy usage RAID5 setup, you'll want to look at a hardware RAID solution.
I dont have a huge amount of experience with RAID-5, but I do know that Software RAID-5 for a home storage PC (Server) works perfectly well under Windows 2003 Server edition. One would presume that it would work just fine under Linux as well. If you want to go RAID-5, then software has one advantage that is often overlooked (mostly because of the disadvantages) - namely that the entire RAID array can be moved to another PC with a totally different motherboard and chipset, all that is needed is the same OS version (certainly with W2K3 Server). I have found that I can have 2 media streams running whilst coppying data from the RAID array to an external HDD on a networked PC without any issues at all, for home use this is more than enough, and most people would be happy with the write speeds (24MB in my case, and thats from a 5400RPM USB drive).


Andy

Nick Geraedts
SPCR Reviewer
Posts: 561
Joined: Tue May 30, 2006 8:22 pm
Location: Vancouver, BC

Post by Nick Geraedts » Mon Aug 27, 2007 5:05 pm

jaganath wrote:AIUI, a pagefile is an area on the hard disk that Windows treats as if it were RAM. in my case, I have a 5400rpm system drive, so it is the slowest part of my system. it does not seem very sensible to store information that is needed quickly on the slowest part of my system, hence no pagefile. also, the amount of HDD seek noise that is created by it is very annoying.
This is another misconception. The page file isn't just "extra RAM on disk". It's where Windows goes to write/read memory when your physical RAM has run out. For more information - read this page. The key part of that is:
Why is there so little Free RAM?

Windows will always try to find some use for all of RAM — even a trivial one. If nothing else it will retain code of programs in RAM after they exit, in case they are needed again.
Disabling your pagefile will not give you a performance boost in any case.
andyb wrote:I removed my Page File from my Desktop on the basis that it will reduce the hard drive seeks and make my PC even quieter, it worked, and along with disabling HDD caching for Firefox I was very happy.

As it turned out I found a performance improvement with no page file and 2GB or RAM that I have been using for a couple of years, I wont be going back.
Again - read the above page. Disabling Firefox HD caching will make a difference, since you're talking about saving temporary files to disk instead of RAM. You honestly won't notice your system "slow down" by re-enabling your page file, but you will notice problems if you system ever does need to use more than 2GB of address space.
andyb wrote: RE: RAID setups.
If you want to go to a heavy usage RAID5 setup, you'll want to look at a hardware RAID solution.
I dont have a huge amount of experience with RAID-5, but I do know that Software RAID-5 for a home storage PC (Server) works perfectly well under Windows 2003 Server edition. One would presume that it would work just fine under Linux as well. If you want to go RAID-5, then software has one advantage that is often overlooked (mostly because of the disadvantages) - namely that the entire RAID array can be moved to another PC with a totally different motherboard and chipset, all that is needed is the same OS version (certainly with W2K3 Server). I have found that I can have 2 media streams running whilst coppying data from the RAID array to an external HDD on a networked PC without any issues at all, for home use this is more than enough, and most people would be happy with the write speeds (24MB in my case, and thats from a 5400RPM USB drive).
Hence why I said "heavy usage". For typical day to day usage - watching a video, making a backup, etc - software RAID5 will be fine in terms of performance. Your 24MB/s limit is the limitation of the USB bus when transferring files - not any of your disks. I've got a 500GB 7200RPM external drive, and it also can't copy much faster than 20-25MB/s. My RAID5 setup (see sig) on a 3ware controller - I can copy files to/from the same array at about 100MB/s, all with no CPU load.

Transferring a dynamic disk (i.e. a RAID array) to another Server 2003 box is possible, but can be tricky and isn't recommended. I've even had troubles with moving a disk from an XP machine to a server 2003 machine - plain NTFS formatted disk. I tried to install it in the Server 2003 box, it it said that the disk had to be initiallized and formatted. I removed it - plugged it back into the XP machine and it worked just fine.

EDIT - One more thing - disabling the pagefile will disable the PerfOS.dll counters, causing all kinds of errors. VMware Authorization won't start properly, Samurize won't be able to collect Performance Monitor data, and so on and so forth. In essence, you're breaking Windows by disabling the pagefile.

http://www.msfn.org/board/index.php?sho ... ntry590288
http://www.samurize.com/modules/smartfa ... p?faqid=18

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Mon Aug 27, 2007 7:28 pm

Cerb wrote:Didn't know Arecas came with another HS. Interesting.
I know, it is not well advertised. If you look in their FAQ there is a response to a question from me about :)
Cerb wrote:2GB VM: 2GB VM is the limit in 32-bit regardless of how much RAM you have. It's a WIndows/IA32 limit. Technically, it can be gotten around with PAE and /3GB, but both cases need the program to be written for it.
Yes, I know, I agree. What I mean is that on a 32bit OS there is a limit of 4GB VM total for the OS. Other 32 bit operating systems can support more than 4GB VM, despite the CPU only having a 32 bit address space, but Windows XP does not support this AFAIK.

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Mon Aug 27, 2007 7:39 pm

Nick Geraedts wrote:What's the problem with having a pagefile anyways? Windows will only use it when necessary (even XP), and don't get me started on the PF Usage graph in Task Manager - that's not pagefile usage!
Disabling the page file will increase performance. Benchmarks tend not to show it, but the system will feel more responsive. The reason is that Windows will page memory out to disk even if there is plenty of RAM available. It makes sense - page out memory for background programs that are idle so that if a foreground program suddenly demands a large chunk of RAM (say Photoshop loading an image) it is instantly available and that app responds faster. It also means there is more RAM available for disk cache etc.

In a system with lots of physical RAM, you can prevent Windows from doing this by disabling the page file. It will eliminate the disk thrashing and delay when switching apps, for example. You can't eliminate it entirely because Windows will still page executable code in DLLs/EXEs/etc, but it does make a noticeable difference on XP.
Nick Geraedts wrote:Regarding onboard RAID controllers - for RAID0 and RAID1, onboard will do just fine.
For RAID 1, software RAID might be better. On better RAID controllers RAID 1 disks can be disconnected and act as individual drives (JBOD) on ordinary controllers if they need to be recovered, but I don't know if the Intel controller can do that. If not, at least with software RAID 1 you can connect the drive to any Windows machine and read it in the event of controller failure.

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Mon Aug 27, 2007 7:44 pm

Nick Geraedts wrote:I dont have a huge amount of experience with RAID-5, but I do know that Software RAID-5 for a home storage PC (Server) works perfectly well under Windows 2003 Server edition.
You can enable it on XP easily too.
Nick Geraedts wrote:EDIT - One more thing - disabling the pagefile will disable the PerfOS.dll counters, causing all kinds of errors. VMware Authorization won't start properly, Samurize won't be able to collect Performance Monitor data, and so on and so forth. In essence, you're breaking Windows by disabling the pagefile.
I have run Samurise without problems. I don't use VMware at home but PM works fine. Maybe your info is out of date and Microsoft have fixed the problem in a patch or something.

Nick Geraedts
SPCR Reviewer
Posts: 561
Joined: Tue May 30, 2006 8:22 pm
Location: Vancouver, BC

Post by Nick Geraedts » Tue Aug 28, 2007 3:30 am

MoJo wrote:Disabling the page file will increase performance. Benchmarks tend not to show it, but the system will feel more responsive. The reason is that Windows will page memory out to disk even if there is plenty of RAM available. It makes sense - page out memory for background programs that are idle so that if a foreground program suddenly demands a large chunk of RAM (say Photoshop loading an image) it is instantly available and that app responds faster. It also means there is more RAM available for disk cache etc.

In a system with lots of physical RAM, you can prevent Windows from doing this by disabling the page file. It will eliminate the disk thrashing and delay when switching apps, for example. You can't eliminate it entirely because Windows will still page executable code in DLLs/EXEs/etc, but it does make a noticeable difference on XP.
From what I can tell, if you look at the PerfMon counter for the pagefile, it doesn't vary drastically when loading new programs - as you might expect it to do when paging memory to disk. Trust me - there's a LOT of misconceptions about what people think Windows does and what Windows actually does. If you don't believe me, run perfmon and keep track of the peak pagefile usage while you use the computer. On my system with 2GB (and a 2GB pagefile) it stays between 2.5 and 3 percent. This is while playing games, using Photoshop, Opera, MSN, VS2005, Outlook, etc etc etc. I've currently got 800MB of RAM free from a total of 2GB (PS-CS3, 4x R@H, Opera, Winamp, and MSN running), and my current pagefile usage is at 1.95%. That ~2% is pretty much universal to all systems I have - some with 512MB of RAM, some with 1GB, and some with 2GB. Simply put - the Windows NT5 kernel (and up) does not page memory to disk unless necessary.
MoJo wrote:For RAID 1, software RAID might be better. On better RAID controllers RAID 1 disks can be disconnected and act as individual drives (JBOD) on ordinary controllers if they need to be recovered, but I don't know if the Intel controller can do that. If not, at least with software RAID 1 you can connect the drive to any Windows machine and read it in the event of controller failure.
This just goes against my principles of relying on an operating system to store the information about your disk. The reason why you can transfer a regular MBR disk to any computer and have it work is because the partition data is stored on disk. With RAID1 in Windows, you need to use a dynamic disk, and dynamic disk data is stored in the OS. You're still running a risk by relying on the OS for that. It's just a principle that I try to stay away from, since you're putting all your eggs in one basket anyways.
MoJo wrote:You can enable it on XP easily too.
Windows XP does not support RAID5 by default. In order to make that work, you need to hack a few files, or replace them with their Server 2003 counterparts, which goes against the XP EULA.
MoJo wrote:I have run Samurise without problems. I don't use VMware at home but PM works fine. Maybe your info is out of date and Microsoft have fixed the problem in a patch or something.
I highly doubt Microsoft would patch this, since they don't recommend that people disable the pagefile in the first place. There are several reports of Photoshop complaining if you don't have a system pagefile in addition to it's own scratch file. Just because it works for you doesn't mean it works in all cases.

The performance boost that you get from disabling the page file is really negligible in comparison with the stability and reliability that you get from leaving it alone. If you really got more performance without it - Microsoft probably would leave it disabled by default on all systems with 2GB or more - but they don't.

andyb
Patron of SPCR
Posts: 3307
Joined: Wed Dec 15, 2004 12:00 pm
Location: Essex, England

Post by andyb » Tue Aug 28, 2007 4:56 am

The performance boost that you get from disabling the page file is really negligible in comparison with the stability and reliability that you get from leaving it alone. If you really got more performance without it - Microsoft probably would leave it disabled by default on all systems with 2GB or more - but they don't.
I have had a couple of problems with a few "retarded" programs that whinge when there is no-pagefile, I got round that one by using a 2MB pagefile which is totally useless but it proves the point that some programs complain about not having a pagefile but dont then complain if its too small and thus useless.

It however was far slower than having a large pagefile or none at all, whether there is or isnt a recordable pagefile size performance difference, not having one on a PC with 2GB of RAM "feels" faster, smoother and more responsive than using one, and of course its quieter as well.

FYI and mostly unrelated, I have set my Azureus Cache limit to 95MB (on a PC with 1GB of RAM) and its fantastic, it saves 90+% of the writes to the HDD, which again makes the PC far more responsive than the default 5MB just because you dont have to wait for the HDD to become available for your own use. This is probably the reason why PC's with no pagefile "feel" more responsive, but isnt recordable in a benchmark - just because the hard drive is waiting for you, not the other way around.


Andy

Nick Geraedts
SPCR Reviewer
Posts: 561
Joined: Tue May 30, 2006 8:22 pm
Location: Vancouver, BC

Post by Nick Geraedts » Tue Aug 28, 2007 12:43 pm

It's well known by all of us that the hard drive subsystem is the slowest part of any computer system. I get around that by making that system as fast as possible (i.e. RAID0). For me, breaking built-in functionality in Windows is just bad. You don't break one thing to fix another.

The mention of the Azureus cache (why not uTorrent btw?) is slightly a moot point. That cache is going to have to be written to disk at some point, so it's either 5MB at a time more often, or a dump of 95MB (or something higher than 5). Another problem with that setup - if the computer crashes when the cache is 70MB full, you've just lost 70MB of data. For some people, that's a long time downloading.

I've played with these settings and "tweaks" for a long time. I used to use nLite to strip down my system to what I thought was "absolutely necessary" - but every time I did, something didn't work. Windows was built the way it was for a reason.

I actually don't mind the occasional soft seek noise. At least I know my computer is working if I can hear the drives when I'm loading my game or PS-CS3 or the like. If I didn't hear the drives, I wouldn't really have any feedback that things are happening (since I don't look at the computer case to see if the LED is blinking).

butters
Posts: 64
Joined: Sun Feb 18, 2007 9:48 pm

Post by butters » Tue Aug 28, 2007 6:51 pm

The only time it doesn't make sense to have a page file is on a 32-bit system with 4GB of physical memory, which is a bad idea to begin with. Since we're talking about a 64-bit system, virtual address space is effectively infinite for the next 30-40 years.

Virtual memory contains three different kinds of pages: working, persistent, and cache. Working pages either map to physical memory or the page file. Persistent pages map to I/O devices and disk files. Cache pages map to copies of persistent pages in physical memory.

If you don't have a page file, all working pages map to physical memory. As long as working memory consumption stays below about half the size of physical memory, this is fine. But if working memory consumes most of physical memory, the amount of space left over for cache pages gets small, and I/O performance suffers.

Because of caching, physical memory is almost always completely full. Whenever we need to make room, the least-recently-used pages are evicted. For working pages, this means they are swapped out to the page file. If there's no page file, the only way to make room is by overwriting cache pages.

So on systems with no page file, relatively fresh cache pages can be evicted because physical memory is filled with really stale working pages. Therefore, systems without a page file are forced to satisfy more I/O requests from disk. It all comes down to keeping the freshest pages in physical memory at all times, whether they're working pages or cache pages. The page file makes this possible.

As for your RAID configuration, the most effective way to RAID your disks is with a non-standard degenerate case of RAID10 (mirror of stripes). One side of the mirror is a striped array of your two 80GB disks. The other side is the 160GB disk. The total capacity is 160GB, and the array can withstand 1-2 simultaneous disk failures.

You won't be able to do this with onboard RAID. I'm not familiar with Windows RAID, but I've done even more unusual arrangements with Linux RAID (three disks on one side, two on the other), so I don't see why you wouldn't be able to do this on Windows.

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Tue Aug 28, 2007 9:38 pm

Nick Geraedts wrote:From what I can tell, if you look at the PerfMon counter for the pagefile, it doesn't vary drastically when loading new programs - as you might expect it to do when paging memory to disk. Trust me - there's a LOT of misconceptions about what people think Windows does and what Windows actually does. If you don't believe me, run perfmon and keep track of the peak pagefile usage while you use the computer. On my system with 2GB (and a 2GB pagefile) it stays between 2.5 and 3 percent. This is while playing games, using Photoshop, Opera, MSN, VS2005, Outlook, etc etc etc. I've currently got 800MB of RAM free from a total of 2GB (PS-CS3, 4x R@H, Opera, Winamp, and MSN running), and my current pagefile usage is at 1.95%. That ~2% is pretty much universal to all systems I have - some with 512MB of RAM, some with 1GB, and some with 2GB. Simply put - the Windows NT5 kernel (and up) does not page memory to disk unless necessary.
Okay, let's try a real world test shall we? Load up Photoshop and Firefox on your 2GB system. Open 30 odd tabs in Firefox to get the memory usage up. Minimise FF and open some files in Photoshop, manipulate them for a while, use some RAM up for undo data etc. Now switch back to FF. Witness your HDD thrashing as it loads FF back in from disk. On my 2GB system without a page file, it just switches instantly.
Nick Geraedts wrote:This just goes against my principles of relying on an operating system to store the information about your disk. The reason why you can transfer a regular MBR disk to any computer and have it work is because the partition data is stored on disk. With RAID1 in Windows, you need to use a dynamic disk, and dynamic disk data is stored in the OS. You're still running a risk by relying on the OS for that. It's just a principle that I try to stay away from, since you're putting all your eggs in one basket anyways.
I don't think you understand how dynamic disks work. All the info is stored on the disk, in the partition data area. There is some info stored by the OS in the registry, but none of it is essential data. It's just stuff like drive letter mappings and metadata. The essential info is on the disk, and you can do things like move one drive of a RAID 1 array to another PC, add another new HDD and rebuild it. That was really the whole idea behind dynamic disks.

Hardware RAID controllers do a similar thing where they store the data about the array on the drives themselves. It is always a good idea to write down the specifics of the array (type, stripe size etc) just in case you need to manually re-build though.

The advantage of software RAID is that if the mobo or controller card dies, you can still reconstruct the array on another system. With proper hardware RAIDs the manufacturers support them for long enough that it's not a problem, but on-board RAIDs often do not. For example, at least with older Promise Fasttrak controllers the mobo controllers are incompatible with PCI ones and the newer mobo controllers.
Nick Geraedts wrote:Windows XP does not support RAID5 by default. In order to make that work, you need to hack a few files, or replace them with their Server 2003 counterparts, which goes against the XP EULA.
Actually you just edit one byte in one file, or download a simple patch program. It does violate the EULA, but so does the rather essential EVD patch you need for P2P programs.

Nick Geraedts
SPCR Reviewer
Posts: 561
Joined: Tue May 30, 2006 8:22 pm
Location: Vancouver, BC

Post by Nick Geraedts » Wed Aug 29, 2007 12:20 am

Regarding your "real-world" test - I ran Opera, PS-CS3, MSN, Winamp, and Dungeon Siege II, as well as four instances of Rosetta@Home. Switching back and forth between the various applications is instantaneous, and there's no noticable delay from memory being paged out. See for yourself:

Image

If I were to switch over to the "Performance" tab, I'd have about 500MB of "Available RAM". Just checking in perfmon - pagefile usage is now up to 4%. The "thrashing" that you speak of is non-existent. I get all of the above mentioned programs back to focus without any large amounts of paging. Maybe your problem is with Firefox and not Windows. Considering all the "tweaks" that people need to apply to make it decently usable, it wouldn't surprise me in the least. Firefox2 has gone down, down hill from where they started.
MoJo wrote:Actually you just edit one byte in one file, or download a simple patch program. It does violate the EULA, but so does the rather essential EVD patch you need for P2P programs.
The "essential" EvID patch actually isn't essential at all. I'm able to max out my 6Mbps connection at home for hours on end, and almost max out the wireless connection to my laptop I've got at my university (There's a 100Mbps pipe through the university, I've hit about 3.5MB/s downloads before - about as high as you're going to go with WPA encryption) all with uTorrent and no patching of any system files. The only thing it limits is the number of half-open connections, which don't really mean anything with a properly configured torrent program.

MoJo
Posts: 773
Joined: Mon Jan 13, 2003 9:20 am
Location: UK

Post by MoJo » Wed Aug 29, 2007 2:27 am

Nick Geraedts wrote:Regarding your "real-world" test - I ran Opera, PS-CS3, MSN, Winamp, and Dungeon Siege II, as well as four instances of Rosetta@Home. Switching back and forth between the various applications is instantaneous, and there's no noticable delay from memory being paged out. See for yourself:
I'm not sure what your screenshot is supposed to prove, but regardless I think this argument has now reached a stalemate. There is no way either of us can prove our points, so we will have to agree to disagree.
Nick Geraedts wrote:The "essential" EvID patch actually isn't essential at all. I'm able to max out my 6Mbps connection at home for hours on end, and almost max out the wireless connection to my laptop I've got at my university (There's a 100Mbps pipe through the university, I've hit about 3.5MB/s downloads before - about as high as you're going to go with WPA encryption) all with uTorrent and no patching of any system files. The only thing it limits is the number of half-open connections, which don't really mean anything with a properly configured torrent program.
Sure, BitTorrent is okay without the patch. I don't think you really understand what it does though - being able to max out your connection has nothing to do with it.

Try running eMule or Share. eMule in particular opens an awfully large number of connetions constantly. With the default XP setting on 10Mb cable I found that even after the initial major rush when opening the program, hours later web pages would time out and VNC connections would fail because eMule was opening enough new sockets to hit the limit (5 per second IIRC). After the patch, all problems cleared up. I noticed them return when a Windows Update patched the DLL again, and I had to patch it back.

andyb
Patron of SPCR
Posts: 3307
Joined: Wed Dec 15, 2004 12:00 pm
Location: Essex, England

Post by andyb » Wed Aug 29, 2007 4:17 am

That cache is going to have to be written to disk at some point, so it's either 5MB at a time more often, or a dump of 95MB (or something higher than 5). Another problem with that setup - if the computer crashes when the cache is 70MB full, you've just lost 70MB of data. For some people, that's a long time downloading.
It doesnt write it in huge chunks every now and then, it writes it in medium sized chunks very regulary. My 96MB or cache is usually 99% full within 30mins of starting Azureus, and if I watch the live updated graphs of To Disk/To Cache, and From Disk/From Cache and look at the total number of To's and Froms I get a very clear cache usage pattern.

Basically I get 10 Cache writes for every Disk Write - so when I am running at 3,174 Disk Writes I am also running 30,861 Cache writes, that is a shitload of Disk writes that Cache has saved the drive from. When I was using the standard 5MB it was about 2-1 at best, 3,000 Disk for 6,000 Cache.

The reads are the other way round I get about 5 Disk Reads for Every 2 Cache Reads - but as my overall Reads are always a lot lower than my writes I have made a huge saving on the work that my drive has to do.

As far as loosing my Cached data is concerned, I dont expect my PC to crash, I hasnt yet and its been in use for 6+ Months, and gets re-booted every 2-3 weeks.


Andy

Nick Geraedts
SPCR Reviewer
Posts: 561
Joined: Tue May 30, 2006 8:22 pm
Location: Vancouver, BC

Post by Nick Geraedts » Wed Aug 29, 2007 2:19 pm

MoJo wrote:I'm not sure what your screenshot is supposed to prove, but regardless I think this argument has now reached a stalemate. There is no way either of us can prove our points, so we will have to agree to disagree.
The screenshot was just supposed to prove that I was using quite a few different programs, and all the while, the pagefile was barely used and there was no disk trashing that I could notice. Just to show that even with lots of "large" applications running, I couldn't notice any difference.

If you want to disable the pagefile on your system, that's fine. I do things with my setups that I don't for others, just because of the "if it goes wrong" scenario. I've started to play with my Windows domain settings, and I've tied down my domain a bit more than some other people (and companies) that I know. I know what I've got to do to fix it, but I'm not always available to fix other peoples' problems if they come up.

Different strokes for different folks. :)


About the disk cache - I've never really been too concerned about the amount of read/writes my disks have had to do - at least from a longetivity point of view. It's not like there's a finite number in the life of the drive. I guess if you're really looking to get that last bit of noise out of the system, then increasing the cache could help. I guess I've just never bothered to minimize the disk usage like that. Again - different strokes for different folks.

Post Reply