When benchmarking is... illegal

The Background

If you ever wondered why this is so difficult to find benchmarks comparing Web Application servers then you are not alone.

I asked myself the question when I was looking for a Web development platform. That was before I decided to write G-WAN.

Part of the answer is where you would never think to look for -in MICROSOFT license agreements (to save time, just read the underlined part):

Microsoft Product Use Rights - Microsoft Servers 
(EMEA)(English) (January 2003) [...]

C. Benchmark Testing.

You may not without Microsoft’s prior written approval disclose
to any third party the results of any benchmark test of
Application Center, BizTalk, BizTalk Adapter for MQSeries, 
BizTalk Accelerator for Financial Services, BizTalk Accelerator 
for HIPAA, BizTalk Accelerator for Suppliers, BizTalk Accelerator 
for RosettaNet, BizTalk Adapter for SAP, Content Management 
Server, Commerce Server, HIS, IIS, ISA, Message Queue Server, 
Mobile Information Server, Project Server, SQL Server, or 
Transaction Server, or any related client software. [...]

D. Other Rights and Limitations for Certain Software.
a. Benchmarking Performance or Benchmark Testing.

You may not disclose the results of any benchmark testing
regarding the Microsoft Server Software, Device Software, 
or the .NET Framework to any third party without Microsoft’s 
prior written approval.

Just in case independent tests would not reflect their own views, MICROSOFT has made sure that nobody can prove them wrong.

Convenient, but also illegal.

The U.S. Dept. of Justice vs. MICROSOFT

Even if the principal anti-trust sanctions were waved by President Bush Jr. some important adjustments were made (to save time, just read the underlined part):

UNITED STATES OF AMERICA,
Plaintiff,
v.
MICROSOFT CORPORATION,
Defendant.

Civil Action No. 98-1232 (CKK)
Next Court Deadline: October 19, 2004 Status Conference [...]

D. Section III.F (ISV Promotion of Competing Software)

Plaintiffs have learned that Microsoft’s contracts for the 
.NET Framework, a component of the Windows operating system 
used to build and run Windows-based applications, require
prior consent from Microsoft before licensees may publish
benchmark testing results for the .NET Framework. 

Benchmark tests can be used to compare the performance of
the .NET Framework to the performance of competing products.

A number of companies offer products based on Java technologies 
that compete against the .NET Framework. For software developers
and distributors, the ability to use such metrics to promote
competing non-Microsoft products would be quite useful in the
marketplace.

Plaintiffs informed Microsoft that they are concerned about 
this restriction on publication of benchmark test results 
without Microsoft’s consent since Section III.F.2 of the Final
Judgments prohibits Microsoft from conditioning the grant of
“Consideration” to any software company based on its agreement
not to promote software that competes with Microsoft operating
system software or Microsoft middleware.

Earlier this week, Microsoft expressed a willingness to modify 
the provision so as to require prior notice to Microsoft of 
various details relating to the testing, but not Microsoft’s
prior consent before publishing benchmarking results.

Plaintiffs are reviewing Microsoft’s response and will make 
every effort to bring this matter to a conclusion prior to 
the October 19 status conference before the Court.

MICROSOFT had to forget about the convenience of illegal censorship.

Today's situation (2010)

Today, one could expect that the matter is settled. Unfortunately, anti-trust sanctions (like all judicial decisions) are efficient only when they are applied.

MICROSOFT VisualStudio 2008's license agreement states that:

SQL SERVER BENCHMARK TESTING. You must obtain MICROSOFT's prior
written approval to disclose to a third-party the results of any
benchmark test of the SQL Server software that accompanies this 
software.

The U.S. Dept. of Justice ruled 4 years before VisualStudio 2008 that the practice (of forbidding benchmarks) is illegal -but this is not preventing MICROSOFT from doing business as usual.

So, what is the message sent by MICROSOFT with this clearly illegal EULA?

Whether you are an end-user, a media or a software vendor:

MICROSOFT is not under any real threat when it breaks the law: past sanctions never only remotely approached the benefits of the fraud.

You may not know about the previous U.S. ruling -even if you live in America.

You may not want to compromize your business relations with the explicit wills of a so powerful corporation that has 94% of the World's market share.

Surprised? That may be because MICROSOFT has made 50,000 'contributions' to wikipedia, hired independent bloggers, journalists and engineers to, among other useful things, "skew Wikipedia articles in their favor".

Quoting MICROSOFT Mind Control (a "confidential" document):

"To control mental output, you have to control mental input.
Take control of the channels by which developers receive information.
Then they can only think about the things you tell them.
Thus, you control mindshare."

MICROSOFT Bing is Windows Internet Explorer's default search engine. With a 94% Desktop market share, Windows is an effective way to make people find only biased information, hiding independent reviews and challengers.

Yes, today's infrastructure is a scam. And yes, we can all do something about it, like voting with our wallets: invest where your money is not used to deceive you.

GWAN