Showing posts with label Barcelona. Show all posts
Showing posts with label Barcelona. Show all posts

Wednesday, July 02, 2008

Anand's Competence Reviewed: Crash And Burn

Anandtech's latest review, AMD's Phenom X4 9950, 9350e and 9150e: Lower Prices, Voltage Tricks and Strange Behavior shows a lot more about Anand's ability as a tester than anything about AMD's hardware.

My older brother used to work on aircraft avionics on the weekends while he was going to college at Purdue. Theoretically he was one of the junior technicians at the small, commuter airline. However, he had gotten in his experience in the Marine Corps working on HAWK missile systems which required an elaborate set of five separate radar units to operate. On one occasion he accompanied a senior technician to another airport where one of their planes was down for maintenance. The other technician worked on the avionics for three hours without success and then quit to go to lunch. By the time he got back from lunch my brother had diagnosed and fixed the problem. I have to say that Mr. Lal Shimpi reminds me a lot of that senior technician. Maybe because I've never heard of anyone or any other review site wrecking systems like Anand does.

But it isn't just destroying systems that is troubling. In this article Anand says:

The first processor is the Phenom X4 9950 Black Edition. Clocked at 2.6GHz, the Black Edition moniker indicates that it ships completely unlocked. Unfortunately the unlocked nature doesn’t really help you too much as the 65nm Phenoms aren’t really able to scale much beyond 2.7GHz consistently

This is a strange claim indeed because everyone else seems to be able to clock these chips to 3.2Ghz with no trouble, and I've seen claims as high as 3.6Ghz. He also insists that Intel's quads will overclock to 4.0Ghz on air. But, in reality when you stress all four cores on an Intel quad they will overheat at 4.0Ghz without water cooling. So, the real difference between recent Intel quads and recent AMD quads when overclocking on air seems to be between 300Mhz and 500Mhz. This means that Anand has taken a probable 500Mhz advantage for Intel and stretched it to a completely fictitious 1.3Ghz advantage. No wonder Anand is seen as a minor deity among the Intel faithful.

This article shows Anand's preferences. He likes Intel. He likes powerful chips. He likes newer chips. And, he likes a low price but he doesn't like giving up anything to get it. This is easy to see from statements like:

The new $133 golden boy, Intel's Core 2 Duo E7200, is actually selling for $129 these days - making it the new value leader from the boys in blue.

He chooses this 2.53Ghz, 45nm chip because he is ignoring the lower priced Allendales that are older, 65nm chips and have less cache. The lower priced Allendales are good chips but since they have less cache and don't overclock as well they are not clearly a better value than AMD chips. The more expensive E7200 is a bargain if you do overclock because it can easily run as fast as Intel's 3.2Ghz E8500 which costs twice as much. Similarly, the Q6600 had enjoyed the status of being the best value quad since it was introduced in 2006. But, Anand doesn't heap the same praise on the 2.5Ghz Q9300 that he does on E7200 perhaps because while Q6600 has dropped to just $210, Q9300 is still running $270. Anand's biggest problem at this point is that AMD is eroding Intel's lead and with it his perceived value of these chips. Keep in mind though that this is mostly in Anand's head; the Q6600 is still a good chip as are the Allendales. The problem for him is that he doesn't want a good chip; he wants a chip that is clearly better than AMD's, and that smug feeling of Intel superiority is getting a lot harder to come by.

For example, you could match the $130 (2.53Ghz) E7200 with an AMD $126 (2.9Gh) 5600X2. Both chips are 65 watt but even at 400Mhz slower the Intel chip will still be a bit faster. The problem is though that Intel motherboards have poor integrated graphics. An equivalent Intel motherboard would need at least a low graphic card to match AMD and you could apply this savings to a $160 (3.2Gh) 6400X2 which with its 26% faster clock is not markedly slower. In other words if you stay with integrated graphics and stock speeds then Intel has no advantage because you will pay more money to get a faster system. However, adding a robust discrete graphic card neutralizes AMD's superior integrated graphics motherboards. And the better overclocking on Intel duals tends to neutralize AMD's lower dual price. This is probably why Anand always pushes a system with overclocking and discrete graphics.

But, things are slowly changing and I think Anand is catching disturbing glimpses of the handwriting on the wall. As the price of AMD's tri-cores gets lower the power of three cores tends to remove Intel's higher clocking dual advantage. Secondly, the B3 stepping greatly improves the overclockability of both AMD's tri-cores and quads. For example, multiply the E7200's $130 by 1.5 for a third core and you get $195. This makes the slightly slower 2.4Ghz Phenom 8750 X3 arguably a good value at $175. And, with its higher pricetag Q9300 is not a bargain unless it can overclock significantly better than 9950 X4. Perhaps this is why Anand is in such denial about how well the newer AMD B3 chips can overclock.

I've also never heard of anyone wrecking so many motherboards during testing. This admission by Anand is a bit shocking:

Let's just say in the motherboard section of the labs that a halon fire extinguisher is now a standard item on the test bench. Call us unlucky, abusive, or having just dumb luck, but our results these past few weeks when overclocking IGP setups has not been good. In fact, it has been downright terrible as of this week.

You see, it is not every week when you can go through five boards in less than 48 hours while trying to make an article deadline.


However, we know that Anand has destroyed motherboards and systems before including systems that were working perfectly before he got his hands on them. This has been going on for a long time, not just this week as he implies. This explains why Anand does not work in a computer repair shop.

Finally, his power testing is typical Anand. He tests completely different motherboards with different integrated graphics and measures nothing but total system power draw. However, he then strangely claims that his results show only differences in CPU power draw:

With the exception of the Q9300, Intel's competing chips draw less power at idle than even the new energy efficient AMD chips

He then pretends he has a fair comparison because he is using top IGP boards from each:

The next set of tests is particularly interesting as we are comparing Intel's top integrated graphics platform (G35) to AMD's (780G). No external graphics card was used, this is strictly an IGP comparison

However, this comparison is laughable since Intel's G35 graphics are considerably less powerful than AMD's 780G and therefore probably draw less power. Anand then makes certain that he covers up Intel's weaker IGP by using discrete graphics cards for games testing. This bait and switch testing scheme is clearly and knowingly deceptive and shows that Anand is not merely incompetent but dishonest as well. Testing based on price, stock clock speeds, stock heatsinks, and integrated graphics would often favor AMD which is why these tests never end up in Anands cherry basket.

Friday, June 20, 2008

The Value Of Benchmarks

Chico Marx had a great line in Duck Soup where he asked, "Who are you going to believe, me or your own eyes?" Apparently Intel is now asking the same question.

Intel has on its website some graphs purporting to show "breakthrough performance and energy efficiency" for Intel 7300 Xeon in virtualization. These are the vConsolidate benchmarks one of which uses VMware. Intel's graph at 2.01 towers over the AMD graph at just 1.08. The problem with this comparison is that reality tends to get in the way. First, Intel is comparing four Quad-Core Intel Xeon X7350 2.93GHz processors against four AMD Dual-Core Opteron 8222SE 3.0GHz processors. Perhaps the fact that Intel is using twice as many cores explains why its score is twice as high. Secondly, where did this vConsolidate benchmark come from? According to Intel:

vConsolidate is a benchmark developed by Intel Corporation to measure Server Consolidation performance.

So we are supposed to trust that Intel didn't massage its own benchmark a bit to favor its own processors? Right. Oddly enough there is a benchmark, VMmark, which is from the same people who make VMware which is what Intel claims to be testing. The problem for Intel is that:

VMmark software is agnostic towards individual hardware platforms and virtualization software systems so that users can get an objective measurement of virtualization performance.

The last thing Intel wants is an objective measurement of virtualization performance when the VMmark results show:

Dell 4x Quadcore AMD Opteron 2.5Ghz 8360 SE R905 - 14.17
Dell 4x Quadcore Intel Xeon 2.93Ghz X7350 R900 - 12.23

With 15% less clock speed the AMD system scores 16% higher.


There are also the SPEC results listed by Heise Online which show:

Dell 4x Quad Opteron 2.5GHz 8360 SE R905 - SPECint_rate2006: 167
Bull 4x Quad Xeon 2.93GHz X7350 R480E1 - SPECint_rate2006: 177

AMD is 6% slower in SPECint_rate with a 15% slower clock.


Dell 4x Quad Opteron 2.5GHz 8360 SE R905 - SPECfp_rate2006: 152
Bull 4x Quad Xeon 2.93GHz X7350 R480E1 - SPECfp_rate2006: 108

AMD is 41% faster in SPECfp_rate with a 15% slower clock.


Not all of the server benchmarks are bad for Intel though. In SAP SD, Intel and AMD are much closer:

HP ProLiant BL685c G5, 4 cpu's/16 cores/16 threads, Quad-Core AMD Opteron 8356, 2.3 GHz: 3,524 SD, SAPS: 17,650

HP ProLiant BL680c G5, 4 cpu's/16 cores/16 threads, Quad-Core Intel Xeon E7340 2.4 GHz: 3,500 SD, SAPS: 17,550

HP ProLiant DL580 G5, 4 cpu's/16 cores/16 threads, Quad-Core Intel Xeon X7350 2.93 GHz: 3,705 SD, SAPS: 18,530

With 4% more speed Intel ties AMD and with 27% more cpu speed it is 5% faster.


While in SPECjbb2005 Intel wins with higher clock speed:

HP ProLiant DL585 G5, 4 Opteron 2.3 GHz 8356s 4 × 4: 368,543

Sun Fire X4450, 4 Xeon 2.93 GHz X7350s 4 × 4: 464,355

With 27% more speed, Intel is 26% faster.


So, if Intel had more integrity they would show the benchmarks where they legitimately win like SPECint_rate, SPECjbb, and SAP SD instead of creating their own skewed benchmarks. I'm sure Intel enthusiasts will leap in to say that the only reason Barcelona does so well is because each of the four processors has its own IMC while Tigerton uses a quad FSB northbridge and has to share the same memory. Interestingly, when I brought up this same point 20 months ago in October 2006 Tigerton or Kittenton? many Intel enthusiasts said I didn't know what I was talking about and that memory bandwidth would not be an issue because the quad FSB Caneland chipset would fix everything. I guess I can't be wrong all the time.

Intel proponents are correct to point out that Nehalem will solve this problem and finally deliver real 4-way performance to Intel. The problem is that this won't happen anytime soon. Today, Intel is stuck with Tigerton and later this year they will introduce the hex core Dunnington which will just make the memory bottlenecks worse. We won't see a 4-way version of Nehalem for more than a year until late 2009.

And, although Nehalem's robust triple channel memory controller has been touted many times the truth is that it isn't needed yet. I've already seen people suggesting that Nehalem's triple channel IMC will increase your gaming performance. Don't hold your breath. The truth is that dropping the FSB and external northbridge does greatly reduce latency. However, in terms of actual bandwidth DDR3 should be fine with just two channels up to hex core. It really isn't until you move up to octal core that triple channel memory begins to shine. Intel already has this with Nehalem so they are ready for late 2009/early 2010 whereas AMD is going to have to finally get the much anticipated G3MX technology out the door to avoid its own bandwidth issues when it goes above hex core in the same time frame.