AMD Frame Pacing Explored: Catalyst 13.8 Brings Consistency to Crossfire
by Ryan Smith on August 1, 2013 2:00 PM ESTThe Test
For the purposes of our testing we’ll be looking at the 6 games we’ve adopted for use with FCAT due to their proven reliability. These are Total War: Shogun 2, HItman: Absolution, Sleeping Dogs, Battlefield 3, Bioshock Infinite, and Crysis 3. All of our results unless otherwise noted are using Catalyst 13.8b1 for the AMD cards, and NVIDIA’s 326.19 beta drivers for the GeForce cards.
Our metric of choice for measuring frame times and frame pacing is a metric we’re calling Delta Percentages. With delta percentages we’re collecting the deltas (differences) between frame times, averaging that out, and then dividing delta average by the average frame time of the entire run. The end result of this process is that we can measure whether sequential frames are rendering in roughly the same amount of time, while controlling for performance differences by looking at the data relative to the average frame time (rather than as absolute time). This gives us the average frame-to-frame time difference as a percentage.
In general, a properly behaving single-GPU card should have a delta average of under 3%, with the specific value depending in part on how variable the workload is throughout any given game benchmark. 3% may sound small, but since we’re talking about an average it means it’s weighed against the entire run, as the higher the percentage the more unevenly frames are arriving. For a multi-GPU setup we’d ideally like to see the delta percentages be equal to our single-GPU setups, but this is for the most part unreasonable. There is no hard number for what is or isn’t right here, but based on play testing we’d say 15%-20% is a reasonable threshold for acceptable variance, with anything under 10% being very good for a multi-GPU setup.
Finally, in our testing we did encounter an issue with Catalyst 13.8 that required we make some slight adjustments to FCAT to compensate for this bug, so we need to make note of this. For reasons we can’t sufficiently explain at this time but has been confirmed by AMD, in some cases in Crossfire mode AMD’s latest drivers are periodically drawing small slices of old frame buffers at the top of the screen. The gameplay impact is minimal-to-nonexistent, but this problem throws off FCAT badly.
To quickly demonstrate the problem, below we have two consecutive frames from one of our Battlefield 3 runs. The correct FCAT color order here is dark blue, green, light blue, and olive. The frames corresponding to dark blue and green occur on frame one, and light blue and olive on frame two. Yet looking at frame two, we see a small 6 pixel high stripe of dark blue at the very top of the image. At this point the dark blue frame should have already been discarded, as the cards have moved on to the green and later light blue frames. Instead we’re getting a very small slice of a frame that is essentially 2 frames old.
The gameplay impact from this is trivial to none; the issue never exceeds a 6 pixel slice, only occurs at the top of the frame (which is generally skybox territory), and is periodic to the point where it occurs at most a few times per minute. And based on our experience this primarily occurs when a buffer swap should be occurring during or right after the start of a new refresh cycle, which is why it’s so periodic.
However the larger issue is that FCAT detects this as a frame drop, believing that over a dozen frames have been dropped. This isn’t actually possible of course – the context queue isn’t large enough to hold that many frames – and analysis shows that it’s actually part of the old frame as we’ve explained earlier. As such we’ve had to modify FCAT to ignore this issue so that it doesn’t find these slices and count them as dropped frames. The issue is real enough (this isn’t a capture error) and AMD will be fixing it, but it’s not evidence of a dropped frame as the stock implementation of FCAT would assume.
Ultimately our best guess here is that AMD is somehow mistiming their buffer swaps, as the 2 frame old aspect of this correlates nicely to the fact that the dark blue and light blue frames would both be generated by the same GPU in a two-GPU setup.
| CPU: | Intel Core i7-3960X @ 4.3GHz |
| Motherboard: | EVGA X79 SLI |
| Power Supply: | Antec True Power Quattro 1200 |
| Hard Disk: | Samsung 470 (256GB) |
| Memory: | G.Skill Ripjaws DDR3-1867 4 x 4GB (8-10-9-26) |
| Case: | Thermaltake Spedo Advance |
| Monitor: | Samsung 305T |
| Video Cards: |
AMD Radeon HD 6990 AMD Radeon HD 7970GE AMD Radeon HD 7990 NVIDIA GeForce GTX 590 NVIDIA GeForce GTX 680 NVIDIA GeForce GTX 690 |
| Video Drivers: |
NVIDIA ForceWare 326.19 AMD Catalyst 13.5 Beta 2 AMD Catalyst 13.6 Beta 2 AMD Catalyst 13.8 Beta 1 |
| OS: | Windows 8 Pro |
Facebook
Twitter
RSS
102 Comments
View All Comments
waldoh - Thursday, August 1, 2013 - link
Its unfortunate it a competing company to shine light on an issue for another to address it.waldoh - Thursday, August 1, 2013 - link
took*tackle70 - Thursday, August 1, 2013 - link
I'd say it's more like expected than unfortunate. This is why competition is a good thing and why you never want one company to blow away another - competition makes all companies serve their customer better.Big time kudos to AMD for their work on this; it's nice to see real competition available again in the $500+ market.
Rezurecta - Thursday, August 1, 2013 - link
Excellent and well said.HisDivineOrder - Thursday, August 1, 2013 - link
I think he was referring to the fact that this issue was present for many years and not only did reviewers not catch on despite common complaints (and HardOCP) discussing the issue, but the company making the card was apparently completely blindsided by it after years and years of Crossfire sales. That's why people who own only one company's cards should try the other side to see that sometimes when someone says something like, "The nVidia cards are smoother in SLI than CF," sometimes--just sometimes--that's not fanboyism. Sometimes, it really is just smoother.No, I think the, "it took a competing company to shine a light on an issue," was more in reference to the fact that nVidia had to basically take AMD by the hand and slowly walk them through how to detect a problem highly prevalent on their products after years and years of waiting for them to get it.
They had to take out their own measurement software they built custom in-house and actually hand it over to the other team just to help them get it. This isn't typical competition teaching the other guy what to do.
This is like Pepsi-Cola taking Coca-Cola by the hand and saying, "Okay, so soda is supposed to have sugar and caffeine. Here is where you get it. Here is our supplier. Try it."
That's why he's saying it's sad. If AMD had figured it out on their own and fixed it, then yeah, that's competition because they FIGURED IT OUT. Instead, they didn't. It took TechReport slamming them on it with DATA after years of HardOCP just slamming them without data and thousands upon thousands of users saying, "Crossfire is not very good compared to SLI" and then nVidia hand delivering them FCAT for them to get it.
Before that, they were clueless. AMD is a company that produces discrete GPU's for the gaming market and not only did they have no clue how to test for this problem, they didn't even know there WAS a problem they were so clueless.
And that truly is very sad.
Galidou - Thursday, August 1, 2013 - link
Not sure that it was as much present in past products, I owned crossfire 6850s for a while then switched to a single 660ti to gain not much except lower temps and a little more FPS. Only game I could tell there was a real noticeable difference in smoothness was Skyrim and that was mainly because of thextures taking more than the mere 1gb my 6850s had.chizow - Friday, August 2, 2013 - link
Can't really agree with this, microstutter was documented and covered significantly in the German press for years, largely ignored by the NA press. 4870X2 microstutter problems were the first time the issue was really brought to light by PCGamesHardware, there's tons of documentation about it about if you search, here's the original test by PCGH:http://www.pcgameshardware.com/aid,653711/PCGH-pro...
Death666Angel - Saturday, August 3, 2013 - link
Multi GPU stuttering was well known about pretty much a few months into having multi GPU solutions. The issue with single GPUs also experiencing uneven frame pacing is much newer. And the believe among AMD was that it was an issue that affects AMD and nVidia equally, which is why they never thought about changing it in their drivers. Until Scott made the revelations.taltamir - Monday, August 5, 2013 - link
I personally documented single GPU multistuttering years ago (caused by lack of CPU power (C2D 8400, problem resolved going to a Q6600; using nvidia GPU), with hard data. (fraps individual frame render times record).I posted it on anandtech forums and there was a brisk discussion of it. It wasn't well known, but it shouldn't have completely blindsided the so called professionals. HisDivineOrder really said it best
chizow - Wednesday, August 7, 2013 - link
Yes I remember, there was a lot of user testing that stemmed from the initial reports on PCGH and the FRAPS frametime methodology became standard in allowing virtually any user who could download FRAPs and work a spreadsheet illustrating microstutter.I do agree though, the pros and press kept ignoring and sweeping it under the rug as if it didn't exist despite countless requests from end-users asking for more detail on it.