How We Test: CPU Gaming Benchmarks

[Matrix]

Today we're discussing a topic that's often raised when we do our CPU gaming benchmarks. As you know, we perform a ton of CPU and GPU benchmarks tests throughout the year, a big portion of which are dedicated to gaming. The goal is to work out which CPU will offer you the most bang for your buck at a given price point, now and hopefully in the future.

Not long ago we compared the evenly matched Core i5-8400 and Ryzen 5 2600. Overall the R5 2600 was faster once fine-tuned, but ended up costing more per frame making the 8400 the cheaper and more practical option for most gamers.

For that matchup we compared the two CPUs in 36 games at three resolutions. Because we want to use the maximum in-game visual quality settings and apply as much load as possible, the GeForce GTX 1080 Ti is our graphics weapon of choice. This helps to minimize GPU bottlenecks that can hide potential weaknesses when analyzing CPU performance.

The problem is quite a few readers seem to get confused about why we’re doing this, and I suspect without thinking it through fully, take to the comments section to bash the content for being misleading and unrealistic.

This is something we’ve seen time and time again and we've addressed it on the comments directly. Often other readers have also come to the rescue to inform their peers why tests are done in a certain way. But as the CPU scene has become more competitive again, we thought we’d address this topic more broadly and hopefully explain a little better why it is we test all CPUs with the most powerful gaming GPU available at the time.

When testing new CPUs we have two main goals in mind: #1 to work out how it performs right now, and #2 how ‘future-proof’ is it. Will it still be serving you well in a year's time, for example?

As mentioned a moment ago, it all comes down to removing the GPU bottleneck. We don’t want the graphics card to be the performance limiting component when measuring CPU performance and there are a number of reasons why this is important to do and I’ll touch on all of them in this article.

Let’s start by talking about why testing with a high-end GPU isn’t misleading and unrealistic...

Yes, it’s true. It’s unlikely anyone will want to pair a GeForce GTX 1080 Ti with a sub $200 processor. However when we pour dozens and dozens of hours into benchmarking a set of components, we aim to cover as many bases as we possibly can to give you the best possible buying advice.

Obviously we can only test with the games and hardware that are available right now and this makes it a little more difficult to predict how components like the CPU will behave in yet to be released games using more modern graphics cards, say a year or two down the track.

Assuming you don’t upgrade your CPU every time you buy a new graphics card, it’s important to determine how the CPU performs and compares with competing products, when not GPU limited. That’s because while you might pair your new Pentium G5400 processor with a modest GTX 1050 Ti today, in a year's time you might have a graphics card packing twice as much processing power, and in 2-3 years who knows.

So as an example, if we compared the Pentium G5400 to the Core i5-8400 with a GeForce GTX 1050 Ti, we would determine that in today’s latest and greatest games the Core i5 provides no real performance benefit (see graph below). This means in a year or two, when you upgrade to something offering performance equivalent to that of the GTX 1080, you’re going to wonder why GPU utilization is only hovering around 60% and you’re not seeing anywhere near the performance you should be.

source