Laying The Ground Work For Proper Testing

I've seen reviews of AMD's Phenom II and Intel's Nehalem. These reviews have varied a lot in quality but none have really provided comprehensive results. It's time to find out.

I originally bought an AMD Phenom II X3 720 Black Edition. Mainly I did this to have something to try out while I was deciding what quad core to buy. The 955 BE looked pretty good. The 965 had a higher base clock but was also more expensive and was rated at 140 watts. On the Intel side, the i7-920 was still much more expensive than the PII 965. But, recently, this all changed. AMD released a new C3 stepping of PII 965 that is rated at 125 watts. Surprisingly, it was released at $200 instead of the $250 that most had been expecting. So, I ordered one. And, I ordered an Asus M4A79X motherboard which is similar to my MA4785 board but without graphics. I also purchased an ATI HD 4650 and couple of ATI HD 5770 graphic cards.

Then I noticed that the i5-750 was the same price, $200, and that I could get an Asus P55 motherboard without graphics that was almost identical to the 79X board and the same $120 price. I ordered those as well. This will give me two almost identical systems. Both systems will be native quad core with onboard memory controller and two memory channels. This should be an excellent head to head, dollar for dollar test. I'll use DDR3-1600 memory rated at CL 8. This makes the most sense because CL 7 memory is still less common, and faster memory rated at 1800 - 2100 Mhz tends to be twice the cost. I'm getting a couple of moderate sized third party coolers to test overclocking although I'm also interested in how much headroom there is with the stock HSF. Moderate sized is close to 500 gram weight, under 130mm's tall, and using 92mm fans. This compares with the heavy coolers which tend to be closer to 160mm's tall, weigh upwards of 700 grams, and use 120mm fans.

The hardware is perfect; this is the closest match of AMD and Intel hardware that I've seen in a number of years. I don't think we've had this close of a comparison since the K7/K8 single core days. The question now is how to test. I'm working on that. My game Dawn of War has a graphic check so see what level is playable. With my IGP 785 graphics the game is only playable with minimum settings. I can check this again with HD 4650, HD 5770 and HD 5770 Crossfire. To be honest, I don't expect to see much difference between the i5 and PII 965 systems. I'll also compare with my X3 720 to see if having another core makes any difference. PassMark has Peformance test which also includes both 2D and 3D graphic tests. I can say that my 785 graphics fail miserably on the last two tests. I'll give these a try but I wouldn't be surprised if they are low enough stress that even the 4650 card passes. I'm hoping that Dawn of War will be require a bit more although it too may top out before reaching the level of Crossfire.

For Integer testing I'm thinking about something based on GMP since this library shouldn't be tuned for either AMD or Intel. Use of the Intel Compiler is obviously out since this would contaminate the metrics. I have Visual Studio but this compiler is only middle of road in terms of what it produces. The next version is looking much better and it is just now available in Beta so we'll see. Better code would be nice but bugs in the Beta version could also contaminate the metrics. However, even the current version should be adequate with Integer code; it is the SSE code that is more of a concern. SSE2 is getting a bit dated. SSE3 is about the minimum level that would be nice to test. Better would be SSE4a versus some or all of Intel's SSE4. I don't have a requirement for full SSE4 testing since a fair bit of this will be replaced with Intel's next upgrade much as SSE became less important as wider SSE functions were added.

My operating system is 64 bit. I'm using 8 GB's memory and see no reason to waste time with 32 bits. Anything that I compile will be 64 bit. I do plan to test with both 2 DIMMs and 4 DIMMs to see if there is any difference. With my system I haven't seen any significant difference in timing or top speed. The two standard cases that I'll be using are both CoolerMaster cases with 200mm fans in front and top and a 120mm fan in the back. I do have a smaller case that only has a single 120mm fan in the back which I could test with. Personally, the notion of putting a $200 processor in a $60 case seems a little goofy but this could show what type of environment would be acceptable. Frankly I wouldn't be surprised if the 125 watt PII 965 were too much for that case.

I'm also glad that I got the X3 720 first since it is rated at 95 watts just like the i5-750. This should give me a pretty comparison of two 95 watt systems although since the i5 is a quad core I would expect it to be more powerful. I suppose if the i5 turned in thermals similar to my 720 while matching the performance of the 965 that would be quite a feather in Intel's cap since it would mean that i5's could be used in smaller cases with less cooling. Overall, it wouldn't be any great victory though since there is no price advantage. Of course, it has been suggested that Intel's power rating is bogus and is actually higher than they claim. Others have tested and insisted that Intel draws less power. Again, I don't really care about previous power draw tests since I have a 95 watt X3 and a 125 watt X4 to compare with. I suspect that since i5 is on the lower end of the Nehalem range it will actually fall in between these two but again I don't know without testing.

And, I have two graphics-free motherboards so I can test without the contaminating effect of integrated graphics. Given the huge gap between AMD and Intel integrated graphics there really is no way to directly compare them. The simplest solution is to discard the integrated graphics and use the same discreet graphic card. For lower level tests or small case tests I would use the HD 4650 which is a pretty solid, middle of the road card. I wouldn't expect a small case with one 120mm fan to be able to handle one 5770, much less two; nor would I expect it to handle an overclocked 125 watt processor. I know that I haven't had any thermal problems with my X3 720 but the case is well ventilated and it is only a 95 watt processor with integrated graphics. If i5 with 4650 does not pass a small case test then I can still project how well i5 would do with integrated graphics by comparing with 720 and the 785 motherboard. And, if the tests are borderline I picked up a heftier, 40 CFM, 120mm Scythe fan which would boost cooling in the small case. This should allow a pretty good inference for cooling with two regular 120mm fans. At any rate, thermal comparisons should settle any question of thermal issues either for Intel or AMD.

I still don't know if my thoughts about case testing are clear. I take issue with the open case, huge cooler testing that they do over at Anandtech. Likewise I take issue with the schizophrenic testing they do over at Toms Hardware Guide. I mean, who in their right mind would put four graphic cards in a small case? I don't really care that much about testing power draw. If electicity is really a concern you could always buy a lower power system with slower memory, 65 or even 45 watt processor and use integrated graphics. But most people are not that concerned about it. Of more concern is whether or not a given case will work with a given system. Generally, everything that people do to increase speed also increases heat. Voltage is increased on the memory, on the CPU, and even on the graphics. Higher clock speeds, more memory, and faster graphics all use more power. We all know that, at various times in the past, thermals were an issue. AMD had K7's before Barton that ran hot, Intel had Prescott which ran very hot and begat the BTX case as a desperate solution. There had been rumors of higher clocked AMD dual and Intel quad core 65nm processors running hot. Of course, now everyone is on 45nm but the top end chips are still rated at 125 or even 140 watts. My results should have much more practical value to people who would like to build lower and midrange systems rather just people at the very top end.

Keep in mind that thermal testing is somewhat separate from performance testing. You can't really begin benchmarking unless you know a given system is reliable. Too often, it seems that reviewers achieve a hasty estimate of maximum clock and then run their benchmarks without really knowing how stable the system is (unless it crashes during the tests). I suppose that and time pressure is why they take so many shortcuts. It takes hours just to run memory stability tests and hours more to run system and stress tests. And, this has to be repeated when trying to find a maximum overclock. Lots of variables like memory voltage, northbridge speed, CPU voltage, base clock versus multiplier all add up to hours and hours of added testing. And this before any actual benchmarks are run. I am confident in the settings on my system. I run the CPU at 3.4 Ghz. I've tested it much higher. I run with auto voltage on both the CPU and NB. I run with the base clock at 200 Mhz and the NB at 2.6 Ghz. I have the memory at 1333 Mhz with CL 7 and 1.545 volts. Auto doesn't work with the memory since auto is 1.5 volts and it will get errors at 1.515 volts with these settings. I'm confident that this is maximum performance for my system. I've tested overclocking the graphics but this isn't really worthwhile since you can get many times better performance without stressing the chipset just by putting in a moderate graphic card like the HD 4650. I expect both the i5 and 965 to show improved performance over my current system.