Conclusion: Round 3?
After looking at the initial Internet battery life results from both tests, one thing is abundantly clear: using live websites with variable content isn't the best way of doing things. One of our original test websites appears to have become less strenuous during the past month or so (perhaps one less Flash ad is showing), and there's always a risk of radically different results if/when a site undergoes a redesign. As much as it pains us to say this, we have to consider all our Internet battery life results suspect at this point. Worst-case, they may be off by as much as 10%.
Going forward, we are working to create some test webpages where the content will be strictly controlled - i.e., identical between test runs. We expect there will always be some slight variation between test runs, but the present variation of up to 10% in some cases is far too high. We could run each test half a dozen times and take the median result, or throw out the high and low scores and average the remaining results. However, running even two tests on each laptop for each test configuration can rapidly result in several weeks of testing and we certainly don't want to triple the number repetitions for each test when they can take several hours to complete.
At any rate, we have started testing with a third set of websites, and hopefully the results will remain consistent for all OSes - and web browsers. We should probably mention that the results in those browser battery life tests are also suspect at this point. We completed those tests several weeks ago, and at the time the results between test runs appeared to be relatively consistent, but we're no longer even 90% confident in those results. We will follow up once we have confirmed our latest testing procedure. Our plan is to start with Windows Vista, perform several test runs on each laptop, confirm that the results are consistent - i.e. no more than 2-3% variation between test runs for a given power profile - and then retest the other operating systems. Once we have tested Windows XP, Win7, and Ubuntu we will reinstall Vista and repeat the tests one last time to confirm that there has been no variation over the course of a couple weeks of testing (or however long it takes). Then we'll think about looking at more browser battery life tests.
Unlike the Internet battery life results, we can discuss DVD battery life results with relative confidence. We use the same DVD, and variations between test runs have always been consistent. We plug in a set of headphones and use Windows Media Player for the DVD playback test. Windows 7 was clearly the winner in this particular test on the Gateway NV52, beating Windows XP by around 6% and Windows Vista by 22%. Again, however, changing to a different platform muddies the waters. On the Gateway NV58, Windows Vista is clearly in last place; XP offers about 21% more battery life and Win7 offers 18% better battery life. That said, this time Windows XP beats Windows 7 by around 3%.
We also said that we weren't going to focus on AMD versus Intel, since we've already looked at that aspect of battery life on these laptops. However, there's no getting away from the fact that the Intel platform offers substantially more battery life. Over 30% more time for each battery test means we only get three or four runs per day compared to four to six runs. (Note that it takes another 90 minutes or so between tests to recharge the battery, and we're not always around to immediately plug the laptops in at the completion of a test.) That means it takes the NV58 anywhere from 4.5 to 6 hours per test compared to 3.5 to 5 hours for the NV52 - and we do need to sleep at some point.
The average increase in battery life over all of the tests we performed so far is 33%, and that's taking into account the fact that Ubuntu closes the gap quite a bit between the two platforms. Throw out that Ubuntu result (only 13% in Intel's favor) and the average battery life lead increases to 37%. Why should we throw out Ubuntu? If you took the time to read this article, you already know that Ubuntu is consistently the worst battery life of the tested operating systems. As much as people like to complain about Windows, manufactures have worked a lot more on optimizing battery life performance for Microsoft's OSes. Then again, as we repeatedly mention in laptop reviews, Apple's OS X is in a league of its own when it comes to battery life. Not surprisingly, having full control over your operating system and hardware can give you a real advantage when it comes to laptop mobility.
The final topic to address is OS performance. Again, we have to pretty much throw Ubuntu out of the running. These are by no means high-performance laptops, but surfing the Internet using Firefox on Ubuntu makes you think you're running an Intel Atom netbook instead of an entry-level notebook. By no means are we discounting Linux in general, and it's still very difficult to beat the price (free); however, we think the vast majority of users will be more than happy to pay the cost for a Windows operating system. Looking at performance on the Windows OSes, once again there are some interesting trends to note. In PCMark05, Windows XP consistently scores higher in every individual test than Vista and Win7; the exception to this statement is the 2D Transparency test, which makes it look like Windows XP is old and outdated. Generally speaking, Windows XP just feels a little snappier than Vista on these laptops, probably due in part to the fact that it has a smaller memory footprint. On the other hand, Windows 7 is the clear victor in general system performance. We would love to have PCMark Vantage results from Windows XP, but unfortunately that's not going to happen. As it stands, Windows 7 outperformed Windows Vista by 15-20% in PCMark Vantage; the individual multitasking test results in PCMark05 also favor Windows 7 over Vista by an average of around 5%.
Without running a reasonable number of game benchmarks, we are not prepared to make any statements about the 3D graphics performance of any of the Windows operating systems. The various 3DMark tests show little to no difference on the NV52, but the NV58 shows differences of 3-19% in the earlier versions and a whopping 84% difference between Vista and Win7. Considering that these notebooks are anything but fast when it comes to gaming performance - IGPs still don't perform very well compared to discrete solutions - and the variations we see may simply come down to driver optimizations. We will leave any firm conclusions about differences in overall Windows performance among the various OSes to the desktop people for the time being. As a laptop OS, we would definitely take Win7 over Vista, but despite UI improvements we still find Windows XP to be more than adequate for most users. Secure? No, but still adequate if you know what you're doing. :-)
At this point, you're probably wanting more information, and admittedly we've only scratched the surface. How do other browsers fare on other operating systems? What about a better Linux distro than Ubuntu? What happens if we use FlashBlock - or a similar add-on - with these other browsers? Can we improve battery life by using a different media player or codec? What about Mac OS X, including differences between Snow Leopard and Leopard - and Tiger, Panther, and maybe even Jaguar if we want to go nuts? Give us time, because if there's one thing we know for sure it's that conducting battery life tests can take a while to complete.
To Be Continued….
Facebook
Twitter
RSS
106 Comments
View All Comments
orionmgomg - Tuesday, September 22, 2009 - link
You are right - I did shouldn't have been so brass on saying who cares - it is a very well written artical and I am sure you spent a lot of time puttinmg together very accurate data for the people who actually do care.Sorry for being thoughtless to say the least.
I always love anandtech and I love all the articals and the diversity of reviews and studies.
I just had a knee reaction to this because every time I buy a lappy - and test the battery for the first time, I realize - yes battery performance gets better and better - even though ever so slightly, but I also realize with great frustration that you really can not rely on battery power for the most part - it will get you out of a jam for sure - but to base any computing time on battery power is futile - that has been my expreiance and so I made those comments more out of my own frustration than anything...
Thank you for all the hard work, good articals and being the very best tech web site!
Fox5 - Tuesday, September 22, 2009 - link
Ubuntu may be released on a 6 month time frame, but by the time a version comes out, the software in it is 6 months out of date. Since Linux is finally getting some real development attention, it's advancing very quickly, and being 6 months to a year out of date represents a large change in features.Hold off on any future Ubuntu comparisons until Ubuntu 9.10 (fresh install only). It switches to the ext4 file system which fixes some firefox performance issues and should decrease boot times as well and should have more up to date drivers. It's also the next long-term release I believe, which is kind of scary considering how much new stuff they're adopting in that version.
I'm surprised you couldn't get the latest ATI proprietary driver installed though. I've used both ATI and nvidia cards in linux, and ati by far had the easier install process. If I recall, it was as simple as sudo ./atiinstaller and then hitting enter a few times. Then again, I've also heard ati integrated chipsets have horrible driver support under linux, so they may not be supported, though I'd think the hd3200 igp would be since it's still fairly recent.
The intel drivers may also get a big boost in ubuntu 9.10. There's some new video driver (or just 3d?) framework being introduced and the older Intel drivers have all but been abandoned to work on the new framework, so a lot of issues that need fixing aren't even being considered for the current drivers in use.
themissinglint - Tuesday, September 22, 2009 - link
This reads like you guys don't know your way around GNU Linux systems well enough to get what you could out of them. It reads like you did the minimum to get it running, which is, from a perspective, fair, since that was probably more work than you put into the Windows OSs.It's also a small sample because you're only using one/two set ups. Laptops like these are optimized up and down for Windows. Of course that is also true for most computers people buy.
Overall, I am glad to see Ubuntu included at all-- it's usually absent from these sorts of tests. The more you include it, the more experience AnandTech will have running Ubuntu (and other Linux systems), and the more you'll be able to get from it. Also, Ubuntu in particular is quickly becoming more out-of-the-box (as more and more people contribute to development on more and more machines). In the next couple years you'll be testing Linux systems that will blow Windows away in some places.
But: Kudos on including Ubuntu... where's OS X? Couldn't you get it to run on a Gateway machine?
JarredWalton - Tuesday, September 22, 2009 - link
Not going to try OS X on a hackintosh... at least not yet. As for the ATI drivers, I went through the manual install procedure multiple times and was greeted with a failed driver message. It was so bad that I couldn't even get back to the original Ubuntu ATI fglxr driver and ended up just going back to my previous Ubuntu image. I did it three times following various instructions and it failed each time. :(If someone has specific experience with getting an optimized driver for the HD 3200 working, point me at some instructions. The experience right now unfortunately was horrible. All the DVD playback programs I tried on Ubuntu crashed when using the latest "stable Ubuntu Jaunty" driver package, sometimes after five minutes, sometimes after 60 minutes. If you weren't at the PC when the DVD playback froze up, the system would eventually enter a completely locked up state (a la the lovely BSOD, but with no error message).
I still have the laptops and will test out other options if any of you want to point me to guides that are of reasonable quality. I got just a brief glimpse of what Ryan experienced over the past year with Ubuntu, though, and I didn't want to get sucked in any further. I have plenty of other reviews I need to get done!
smitty3268 - Tuesday, September 22, 2009 - link
I think that crash during video playback is fixed by turning off redirected rendering when full screen apps are active (it's an option in Compiz somewhere). Or you could just turn off desktop compositing completely. It seems like the newer drivers have fixed that, but I'm not running fglrx and I don't remember for sure.Fox5 - Tuesday, September 22, 2009 - link
Agreed, the desktop effects, besides wasting battery life, interfere with the ATI's drivers video acceleration and should have been off during the test.Of course, you could then ask why Ubuntu enables something buggy by default.
JarredWalton - Tuesday, September 22, 2009 - link
I did disable Visual FX on both systems - is that the same thing? The ATI setup was pretty much a complete no-show with DVD playback before I did that. It would stutter and fail withing seconds. Sorry I didn't mention that.nofumble62 - Tuesday, September 22, 2009 - link
Intel battery life at least 1 hour longer.AMD boot up time 50% longer.
Those are the two most important things for me.
jodomcfatty - Tuesday, September 22, 2009 - link
If you look at the specs, he's comparing a Intel 4500 graphics chip with an AMD 3200HD(which while integrated, will seriously destroy the 4500 in gaming tests) but that will ALSO completely affect battery life since it can't just be turned down to 4500 usage levels. This thing alone eats up significant battery power while expelling a lot more heat.He did this before and I completely lose respect for him for talking battery life while using totally different setups.
I would agree that equal systems the Intel will most likely be better, but no where near by this much.
This article is more about the OS but even there he knows little about linux but I will say he's willing to do more testing....just seems crazy to me that he couldn't just get a AMD with a 4500(harder to find but not impossible)
JarredWalton - Tuesday, September 22, 2009 - link
What's a better IGP for AMD than the HD 3200? You're being completely obstinate to suggest that it's not a fair comparison.The fact is, AMD can't do any better power-wise than what I've got in the Gateway, can it? An X1270 IGP is similar in performance to the 4500MHD, and power draw is similar to the HD 3200 (because it's on an older process).
As for finding an AMD setup with a 4500, either you're talking about an AMD HD 4500, which would be even WORSE on power draw than the HD 3200, or you're suggesting it's possible to run an AMD CPU with an Intel chipset. It's not, as HyperTransport isn't an Intel feature and that's what AMD CPUs use.