Test Setup
As stated, we are testing four different operating systems. In order to keep the number of benchmarks manageable, we are focusing on two primary battery life scenarios. Our first test is an Internet web-browsing scenario. We configure the chosen browser to load three websites that contain Flash content every 60 seconds. When the battery level goes critical (usually 1%, although Windows 7 doesn't let us set this lower than 3%) the system shuts down. The second test is DVD playback battery life, again with the system set to run until the battery level is critical. The various operating systems don't necessarily give you the same level of control over power saving features, so we tested a couple scenarios on each Windows OS.
Windows XP
Windows XP gives you the least control over power saving features. We test two scenarios, one using the Portable/Laptop profile and the other using the Max Battery profile. We don't have specific details on what these settings mean in terms of maximum CPU performance, but it's clear that they do affect performance. They also impact battery life, and most people only want enough performance when running on battery life for their system to work properly. We did not experience any difficulties watching DVDs or surfing the Internet, although more CPU intensive tasks may have problems when using the Max Battery profile. On the Portable profile, the HDD is set to power down after five minutes, while the Max Battery profile has it set to power down after three minutes. We also measure LCD brightness with a colorimeter and set it as close to 100 nits as possible.
Windows Vista/7
Similar to Windows XP, we test two different power profiles in Windows Vista and 7: Balanced and Power Saver. These profiles correspond roughly to the same settings as the Windows XP profiles, but this time we have more control over the various details. For Windows Vista and 7, we use the following settings under Balanced.
The hard drive is set to power off after five minutes. The wireless adapter is set to medium power saving, hybrid sleep is enabled, and USB selective suspend is enabled. PCI-E link state power management is set to maximum power saving. The minimum processor state is set to 5% and the maximum processor state is 100%; we don't know how this specifically affects CPU clock speeds, and if you want to run a similar test make sure your notebook manufacturer doesn't specify different values. Search and indexing is set to power saver and adaptive display is disabled. Again, we calibrate the LCD to run at 100 nits, which is 35% brightness for these two laptops (three steps above minimum brightness).
The Power Saver profile uses the same settings in most areas, but we set the hard drive to power down after three minutes, the wireless adapter is at maximum power saving, and the maximum processor state is 50%. Windows 7 does include a few other areas that you can tweak, but most of these relate to LCD dimming in order to conserve power. This is not to say that Vista and Windows 7 are the same, because as we will see in the results there are some definite differences.
Ubuntu 9.0.4 (Jaunty)
There aren't a lot of power saving features available for tweaking within Linux - at least not that I could find. We configured the LCD to never shut off (just like on Windows), and set the system to shut down at 3% battery - the minimum we could specify. We also disabled LCD dimming. Honestly, I don't know nearly as much about Linux as Windows, but I did my best to get Linux installed properly on the two notebooks. As anyone who has used Linux before can tell you, getting driver support is sometimes a real pain in the rear.
Generally speaking, both systems also felt far more sluggish running Linux, especially for typical web browsing. We used Firefox 3.0 initially and then updated to 3.5 - not as easy as I would like under Ubuntu, and I have no idea why the latest build is called Shiretoko in some places and Firefox 3.5 in others. Whatever. Both versions of Firefox felt incredibly slow, but that was with Flash enabled. With the FlashBlock add-on, Internet performance was significantly better, but that isn't a fair comparison to the Windows browser tests. As a point of reference, we ran the same test under Windows Vista and achieved significantly better battery life. We will get to those results in a moment.
Our issues with configuring Ubuntu don't end with the browser, however. On the Intel-based NV58, we could not get LCD brightness to function properly. The result is that the LCD ran at maximum brightness during our tests, which obviously results in lower battery life. We tried to find an updated driver for the Intel GMA 4500MHD graphics, but so far we have been unsuccessful in addressing this issue. The AMD-based NV52 wasn't any better, but for different reasons. We could find updated drivers from ATI, but after repeated attempts we never did get them to function properly. We were left with running the proprietary fglxr 8.600 driver, and while it worked fine in general we had problems with DVD playback. VLC repeatedly crashed during our benchmarks, sometimes after a few minutes, sometimes after 50 minutes. Eventually, we decided to uninstall the proprietary ATI driver and test out the open-source MESA driver. Surprisingly, the open-source driver actually provided a better experience, although we aren't looking at 3D performance where ATI's fglxr driver likely has an advantage.
The short story is that while everything eventually worked out okay on the NV52, the number of headaches we encountered trying to get everything working properly under Ubuntu is more than a little frustrating. You get a sense of accomplishment once everything is working, or at least I did, but that's not something most people are interested in doing. Most people view computers as a tool and they wanted to "just work"; they don't want to scour the Internet looking for instructions on how to manually install the latest drivers/application/whatever. If you're Interested in a more detailed look at Linux, we suggest you read our Ubuntu 8.04 Article. The only area of Linux that we're testing today is battery life.
Facebook
Twitter
RSS
106 Comments
View All Comments
nortexoid - Wednesday, September 23, 2009 - link
I'd like to see a test done using "regular" office apps (openoffice, acrobat reader, etc.) and NO internet browsing. (Yes, that's how I use my computer off the plug usually.) It would weed out the performance usage hit taken by Ubuntu when flash is running.It might also be interesting to do a test with tweaked systems, e.g. by tuning Jaunty with PowerTop or similar apps. How does each OS perform when FULLY optimized for battery life (without sacrificing features or much performance, of course)?
7Enigma - Wednesday, September 23, 2009 - link
I know there are users (Jarred apparently you are one of them) that run a system without antivirus/spyware, but you are (or at least should be) in the minority. Linux distro's apparently can get away without it, but on ANY Windows box it's a MUST HAVE. Also disabling those other services (while good at reducing variables) again undermines the system's protection and comparisons to a general usage scenario.In your article I do not think you mentioned what you tweaked (but I could have overlooked this). From my initial impression when reading the article you took both OS' as they were default installed and then tested from there. I think you skewed the results badly in the favor of the Windows platforms by doing this, and I say this as a Windows-only user (never used Linux in any flavor). The first thing I do after a fresh install (still on Vista) is turn off all of those programs you did, along with a host of other services/eye candy using BlackViper's Vista Tweaks. This significantly speeds up the OS in all aspects but can hardly be considered most users configs.
At first reading these comments I was firmly on the "sour grapes" to all of the Linux users complaining about what distro was used, or why X wasn't tweaked by going to a website and reading a tech article, but now I kind of agree with them. Your experience with Windows and lack of experience with Ubuntu had you setting up one for failure before the first test was even run.
Either you test both OS' as they are default installed (driver incompatibilities aside), or you need to have a Linux semi-guru set up your Ubuntu box. My recommendation is the former, as the latter has so many variables it's probably not worth testing in the first place.
Aside from the Ubuntu portion I enjoyed the article. It was very interesting to see how the different power profiles jockeyed for position. I would like to second an earlier commment that asked for idle time to shutdown numbers. Let's be honest, most of us are not going to surf continuously from 100% to dead battery. Rather we are more likely to use the laptop for a bit and then walk away, and come back later. It also will give a good indication of those background tasks impact on battery life (if you don't disable them in your config). I have a sneaky suspicion XP might look very good as it seems to be quite a bit more bare-bones than Vista/7.
Thanks again.
code65536 - Tuesday, September 22, 2009 - link
An Intel engineer explained it some years ago in a blog posting: XP's max battery basically throttles the CPU's frequency down all the time. For example, on my Core2Duo, it'd mean that the CPU will operate at 800MHz all the time, even when it's busy. On the other hand, if Windows gives Speed Step a free hand and lets Speed Step determine the speed, then the CPU will operate at 2GHz when busy, and slow to 800MHz only when it's idling. According to this Intel engineer, it makes no sense to have it always throttled to 800MHz, because that means that tasks will take longer to finish, and the CPU will spend less time idling (which is when the CPU *really* saves power), and he labeled the max battery mode as the "what the hell are you doing?" mode.Drizzt321 - Tuesday, September 22, 2009 - link
I completely understand your complaints on the length of time required to run the test, plus recharge the battery. Have you considered buying a 2nd or 3rd battery and finding an external charger? Or would that kill the results of the test because the battery would be different?PepperPot2 - Tuesday, September 22, 2009 - link
I fail to believe you included the default grub time in the boot as this to do so would be unrepresentative of real use. Who would sit there and wait for it to time out rather than just press the enter key triggering the boot process? Anyway a default install of ubuntu (where it is the only OS on the machine) doesn't show the grub menu, only a 2 second alert to allow you to bring it up if you want. I then immediately boots.My experience with ubuntu is totally contrary to the conclusions you just posted about speed. We've have ~20 installs of Ubuntu 9.04 at work on old machines (7 yrs old) to a machine I built 1 month ago. In all instances the staff find it it is smoother and quicker to use than the dual boot copy of vista or XP.
The boot time on XP is clear nonsense, when ubuntu boots to the desktop you can use it almost immediately. XP is typically totally unusable for ~30 seconds while it loads more crap in the background. Vista is worse.
jasperjones - Tuesday, September 22, 2009 - link
That's true, XP and Vista are pretty unresponsive for a minute or so after boot on your average entry-level laptop. OTOH, Ubuntu is snappy once you see the GNOME desktop. 9.04 is called jaunty ducy ;)However, the same can be said of Win 7. It keeps loading stuff after you see the desktop, but that doesn't make the system feel sluggish--not at all
JarredWalton - Tuesday, September 22, 2009 - link
The Windows boot times are stopped when I see the default system try icons appear (WiFi connectivity, volume, etc.) so the system is pretty much usable at that point. As for GRUB, it's a 2 second timeout, which I could bypass by pressing enter twice, but that still leaves around 8 seconds from the POST to the point where it looks like Ubuntu is actually loading. What's going on during that time? If we subtract POST times, then Ubuntu would look even worse. I can see about tweaking GRUB to skip straight to loading the OS and see if that helps as well... when I get time.jasperjones - Tuesday, September 22, 2009 - link
FYI--one can simply configure GRUB to not wait for user input by editing /boot/grub/menu.lst and changing the second or third line from "timeout x" to "timeout 0"But I generally completely understand the argument not to change too many things from the default installation of Ubuntu, as it's doubtful that the average user would do it.
maveric7911 - Tuesday, September 22, 2009 - link
I would love to see a properly installed and setup distribution of linux in this review (not bashing love that you included linux at all). As others stated above Archlinux would be a pretty good choice. All packages come optimized and things like native 64bit flash and other native 64 bit applications are apart of the stable repository "no adding repos necessary". Also as mentioned, not using the hardware accelerated driver of ati or nvidia will take a big hit on power right away. I'm always available to assist with any linux testing and/or questions.HP
Sr Linux Engineer
andrewaggb - Tuesday, September 22, 2009 - link
What I would like to see is Jared run all the tests on his version of Ubuntu, then let a linux expert have some ssh love to tweak it all up, and then let Jared run the tests again and see how much difference he finds.