Let me say that it is not my intention to demonize Microsoft on this issue, because this is not just about them. In fact, I was delighted to see Murchie's letter because at least it meant a vendor was going on record about this policy. You may remember that a few weeks ago Network Associates refused to comment on the rationale behind a term in its VirusScan click-wrap agreement prohibiting publication of benchmarks and reviews without company permission. And Murchie is right that Microsoft isn't the first database vendor to try to enforce this restriction. Oracle has been threatening customers and publications with its benchmark restriction for more than a decade and in some cases has succeeded in squelching reviews.
But Murchie says that Microsoft's benchmark restriction is not meant to prohibit customers from sharing benchmark information with each other, which they can do without getting Microsoft's permission. As written, the SQL Server license itself would seem to apply to customers as much as to publications, but Murchie told me he plans to advocate changing the language to make it clear that it applies only to publishing benchmarks.
All that said, what Microsoft has done in this case is a very big step down an extremely slippery slope. Perhaps Kennedy's results were inaccurate, but Microsoft never proved they were. And who gave database vendors the right to judge whether a benchmark of their product is accurate?
That's what really worries me about Microsoft's actions here. After all, Kennedy wasn't benchmarking the performance of two databases; he was just using a database as part of a test comparing two versions of Windows. Think about the implications of that. The Windows licenses don't have the benchmark restriction, so will Microsoft now decide it needs that language in all of its end-user license agreements? And will that mean that anyone wanting to test the performance of a product that uses Microsoft software be required to get Microsoft's blessing first?
Taken to its logical conclusion, Microsoft's claim that it has the right to keep Kennedy from disclosing his results is tantamount to saying the company can keep Road&Track from testing the performance of a car that has a little bit of Microsoft software embedded somewhere.