April 17, 2001, 1:18 PM — Speak now, or forever hold your peace. Because the next time you try to speak, you may need Microsoft's permission. As Cringely readers are already aware, Microsoft recently prevented an independent lab from publishing benchmark results by using a term in the SQL Server license that says the user "may not disclose the results of any benchmark test ... without Microsoft's prior written approval" to threaten the lab with legal action.
The incident was mentioned by my esteemed pagemate in several columns and reported in detail by John Fontana in Network World, March 12 (see "Microsoft gets tough with independent testers"). InfoWorld's March 26 issue then carried a letter from Steve Murchie, SQL Server group product manager at Microsoft, explaining his company's position.
To summarize the tale briefly, independent lab Competitive Systems Analysis (CSA) of Danville, Calif., recently ran a series of tests based on SQL Server to compare the performance of Windows 2000 vs. Windows NT. We don't know the results except that CSA Research Director Randall Kennedy was surprised enough by them that he thought he must be doing something wrong. Microsoft engineers worked with him for five days to find the problem without success. Then, knowing that Network World was interested in including Randall's results in a story, Microsoft officials trotted out the no-benchmarks term from the SQL Server click-wrap agreement. Kennedy briefly considered defying them but decided not to get in a legal fight with Microsoft over tests that no one was paying him to do.
Microsoft's Murchie says the only purpose of the benchmarks restriction is to ensure that the public gets accurate information. It's standard practice in the database industry, he says, where vendors have worked together to develop standardized tests with repeatable results, and Microsoft is not the first database company to block publication of benchmarks.The license language is intended to keep labs and publications from rushing to print what might be misleading information and to prevent scurrilous competitors from using finagled test results in their advertising.