Is it Reporting or is it Marketing?
The time has come, the web-man writes, to rank their offerings. With buffer chips, performance scores, capacitors and things.
In the tale that the above quote has shamelessly butchered, the Walrus takes the guise of a ‘guru’ and uses clever banter to lure the unsuspecting oysters out of their beds so he can eat them. While the various hardware sites are not intending anything so devious, they sometimes do present misleading information or just plain misinformation as fact under the guise of ‘hardware expert’. This leads some novices to make hardware choices based upon some criteria that I consider to be marginally important, at best.
Lest I be misunderstood, I believe that these sites do provide a valuable service, and I have a great deal of respect for the webmasters. In fact, I have a good relationship with most of them. I just think that it is time we understood that there are some limits to their expertise, and that sometimes they can be mislead by marketing BS just like anyone else. Few of these webmasters are computer scientists, electrical engineers or even practicing computer professionals!
The following is not a complete list of errors, nor is it meant to be. I have picked the issues that I feel are the most grossly overrated in terms of criteria to choose a motherboard and have provided my reasoning and opinions about them. Note that reasoning and opinions are considered separate lines of thinking ;-)
TI Data Buffer
Intel has made a suggestion that motherboard manufacturers use data buffer chips for stability with 8 banks of system RAM. While they don’t indicate it is absolutely necessary, it is being used by some manufacturer’s as a selling point. The fact is that it is possible to achieve stability without using this data buffer by designing the circuitry correctly. Unfortunately, this feature is being used as a criteria for determining the stability of a motherboard without actually determining whether the boards without it is unstable or not!
One manufacturer’s marketing announcement says that “No one knows yet if the 440BX boards will operate perfectly without it… We feel you are taking your chances if you buy another motherboard without this 6 chip memory data buffer enhancement. When it comes to system crashes we don’t think you should gamble and reliability is our primary concern…” Of course, if the motherboard has been tested with 4 modules and has proven to be stable, then there really isn’t much of a gamble at all!
I have personally tested one manufacturer’s BX motherboard (AX6B) which does not have this data buffer, and it is an exceptionally stable motherboard with all 4 memory slots filled. This board has also been tested by several memory manufacturers who have told me directly that the AX6B is one of the most stable BX boards they have tested. At least one review site has noted that another manufacturer’s board without these data buffer chips also works fine with 4 DIMMs installed.
When I made this statement on usenet, one individual replied that “the data buffer isn’t necessary, it just makes the board more stable.” Apparently, he didn’t even read the entire comment. One might make the argument that if you didn’t know how stable either board is, you should pick the one with the data buffer. Of course, if you don’t know how stable either board is, you should go do some research and find someone that has tested it. After all, what is it you are looking for at these hardware sites – speculation or actual test results?
I personally believe that the inclusion of the data buffer may actually slow down the memory transfers, and therefore may affect performance somewhat. It would be interesting to see a comparison of the memory transfer rates between a set of boards with and without the data buffer. My opinion is that the data buffer may make it easier for engineers to design the circuitry, but an engineer that knows his stuff shouldn’t require it. Perhaps at very high bus-speeds (such as 133MHz), the inclusion of the buffer chips may prove beneficial, but I would need to see some evidence of that.
Elecrolytic vs. Tantalum Capacitors
I must admit that this one really gets to me. I have had people tell me that the reason they picked on particular motherboard over another is because it had Tantalum capacitors, and the other one had Electrolytic capacitors. The actual names for these are Aluminum Electrolytic and Tantalum Electrolytic capacitors, though a dry Tantalum capacitor is not truly electrolytic.
Several years ago, a few major publications (EE Times, Computer Shopper, etc.) ran some articles on this subject. At that time, the ‘no name’ Taiwanese motherboard manufacturers were flooding the market with cheap, poorly made motherboards that threatened the market share of the larger manufacturers. In order to combat this, a marketing blitz ensued in which they concentrated on the one component that would be easy to identify – the capacitors. The cheap boards used very inexpensive Aluminum electrolytic capacitors, while the better ones used high quality electrolytic capacitors along with a few Tantalum capacitors. Since the average end user couldn’t tell the difference between low and high quality (obviously), these high-end manufacturers decided to concentrate on the Tantalum vs. Aluminum issue – and obviously succeeded far beyond their expectations!
In the original articles (which you can find by searching for the words ‘Capacitors’, ‘Tantalum’, ‘Electrolytic’, etc. on the major search engines), spokesmen for these manufacturers pointed out that very cheap capacitors can lose their capacitance after a year or so, leaving your expensive components (processors and memory) unprotected from voltage spikes. They mentioned that only a few Tantalum capacitors were really necessary, and that at least 15 total capacitors should be present. What they didn’t say was that a high-quality Aluminum Electrolytic capacitor may be much better and longer lived than a mid-grade Tantalum capacitor. They also didn’t mention that there are other considerations in choosing a capacitor, such as low ESR values.
If you are not familiar with how capacitors work, you can get a good idea by thinking of electricity as water and a capacitor as a Dam. Electricity is stored in the capacitor, and released at an even rate. By putting capacitors in a circuit with a sensitive electronic component, such as memory or a CPU, that component can be protected from power spikes. Though Tantalum capacitors are generally considered ‘better’ for many applications, there are very high quality Aluminum electrolytic capacitors that can last for up to 20 years without losing their capacitance, and have lower ESR (equivalent series resistance) values than the equivalent Tantalum capacitors.
AOpen has a discussion in their motherboard FAQ about why they have chosen Aluminum Electrolytic capacitors over Tantalum. They have indicated that they chose these components because of the lower ESR values while not sacrificing longevity. What is especially irritating is that on some of the hardware websites, the comments about electrolytic capacitors seem to imply that these non-engineering, non-degreed, non-professional webmasters know better how to design a motherboard than the actual professionals who are working in the field! Buyer Beware! This is one of the most ludicrous criteria with which to rate a high-end motherboard that I can imagine.
Ziff-Davis has successfully convinced people that they are the undisputed benchmark kings. Unfortunately, they don’t tell you many things. For example, they say that they have grouped the Business Winstone tests in to ‘groups’, but they don’t tell you what their weighting is in determining the final score. They also don’t tell you what the accuracy of their results are, other than to mention that you shouldn’t get a variance of more than a ‘few percentage points’ between runs on the exact same system.
The Winstone tests are intended to be very generic in nature. Rather than produce a set of benchmark test that are specific to an industry (i.e. – architects vs lawyers vs real estate) they have chosen only the most popular ‘business’ applications such as spreadsheets, word processing, presentation managers and database managers. The reasoning behind this is obviously that for a minimal cost they can produce a benchmark that has meaning for the ‘majority’ of users. While this may work fine for testing complete systems in a fixed configuration, it breaks down when testing specific components.
Anyone who is paying attention can see that benchmark tests between motherboards with the same chipset are usually within one percentage point of each other. This is well within the variance of the Winstone tests, and should be interpreted as equivalent scores. Unfortunately, the hardware websites not only publish the ‘exact’ number, but scale their comparison charts such that a 1% difference looks like a 20% faster motherboard! When looking at these charts, note where the scale begins and then try to picture what it would look like if the chart were scaled at 0. Most likely you wouldn’t be able to tell any difference in the lines were this done. What this means is that though the hardware sites are presenting themselves as objective, rational reviewers they are in fact engaging in marketing tactics by showing pictures that exaggerate the actual differences.
Something else to recognize in these benchmark tests is that they are heavily dependent upon I/O speed and Video Speed, as well as the speed and amount of memory. By simply changing a video driver, you can affect the results significantly. In fact, it is entirely possible to increase the Winstone scores by over 30% merely by swapping a low end video card with a very high end one. Even changing hard drives, adding another stick of RAM or changing some BIOS settings can affect the numbers. Most people will never achieve the numbers presented by the hardware sites, simply because they are using the fastest components they can get their hands on at the time (remember – this is not a job or professional publication for most of these, it is a fun hobby!).
Once you accept the fact that quality motherboards made from the same chipset will have the same benchmark results, the rating of these boards based upon ‘speed’ becomes ludicrous and should not even be included in the ratings. Unfortunately, you will see these sites give a higher mark in this area because one motherboard posted a .5 Winstone point difference (less than 1%), and can sometimes be the difference between the #1 and #2 rated board! In actual fact, you should decide which chipset you want (this determines what features are possible), then choose the motherboard based upon other factors
The Overall Ratings
There are hardware sites that provide overall ratings for motherboards, and use those as ‘comparisons’ between models, manufacturers, etc. When looking at these, be careful about the numbers. Most of them are not based upon fact, but upon opinion or even complete conjecture.
For example, at least one hardware review site lists ‘Reliability’ as a criteria, and actually gives a rating. How on earth can a person using *one* sample determine the reliability of a product? Reliability means how well it will work across a range of applications and environments as well as over time. RMA rates are a big factor in determining reliability, however these review sites are virtually incapable of knowing this information – especially when reviewing pre-release products.
Most of the hardware sites make at least some reference to quality, stating that by looking at the board they can determine it’s overall quality. Unfortunately, they are expressing an opinion, even if it might be an ‘informed’ one. When discussing quality in regards to a product, one is referring to the overall quality of a population, not one specific item. Quality is determined by testing, not by simple observation. If a product is able to run every application, is compatible with every add-on component and does not break over time it can be considered high-quality. When you have an entire production line of products that exhibit these qualities, you have a quality product. Any individual item can be either great or junk, regardless of whether the population in general is high or low quality. It is very unlikely that any of these reviewers can truly determine the quality of a motherboard merely by testing one sample with their limited set of components, applications and time spent testing.
One other issue to recognize is that most of the ratings (other than pure performance numbers) are mostly opinion rather than any scientifically derived number. If these reviewers had oscilloscopes where they could actually see any circuit related problems, they had a standard ‘stress test’ that all boards would be subjected to or they had a ‘burn-in’ period of at least several days, I might be willing to accept that there is some credence in their numbers. As it is, I believe it is mostly opinion, predjudice and guesswork in deriving these numbers. Every person has some bias towards or against a particular manufacturer for various reasons. These biases can change over time, depending upon our experiences. We all should recognize this and cast a wary eye upon the sites that try to pass off their opinion as some scientifically determined number.
How Do You Choose?
Unfortunately, what all this means, in my opinion, is that users must do much more research when choosing a motherboard. Don’t just blindly use one review site’s comparisons. Check all review sites and ask for opinions amongst users. In fact, user’s comments are actually better information than the review sites, generally. Of course, if the review is for a pre-release product you have limited abilities to get this kind of feedback. In those cases, you need to weed out the BS from the facts. See what hardware components were tested with the motherboard and what actual tests were performed. If only one set of components was used, and only Winstone tests were run any rating other than performance is highly suspect. Compatibility, Reliability and Quality cannot be determined this way and the numbers should be ignored if they are given.
As a point of reference, we always test new products with a low-level hardware diagnostic card which tests down to the circuit level (called a P.H.D Plus card). We then test modules from various manufacturers in different combinations, different video cards and sometimes different hard drives. We also test multiple samples if available (we buy our samples, we don’t get them for free usually) and track RMA rates over time. This provides a much better measure of compatibility, reliability and quality.
I don’t expect that this will change the beliefs of most people. For some reason, people like to choose their ‘expert’, and then religiously defend him/her without questioning. Usually this is because the individual does not fully understand how to evaluate the products and must put their faith in someone else to do it for them (whether it be a friend, web ‘guru’ or local technician), and has to feel that they have made the right choice. What I would hope is that people will look at many of these reviewers and evaluate their methods before deciding if their recommendations can be blindly followed. By providing feedback and asking questions, we can make all of the review sites even better than they currently are.
Be the first to discuss this article!