Benchmarking online performance

Many people involved with web analytics seek benchmarks for their online performance. They want to know how their performance compares with others in order to know if they are doing well or not. Web analytics consultants are familiar with the request for benchmarks, it is something they get asked for frequently. Unfortunately it is not something they can provide.

There are no benchmarks for online performance. There are good reasons for this, which this article will explain.

What is Benchmarking?

Benchmarking, as it is used in web analytics, can refer to comparing your own performance with the overall sector or it can refer to comparing your performance with relevant best practice. Comparing with best practice is the original use of the term in management circles. The idea is that by comparing aspects of your performance with industry standards or with industry leaders you can determine what you are doing well and what needs to be improved.

Why Do Benchmarking

The primary reason people seek benchmarks is to see if they are doing well or not. Most commercial websites are complex structures, and the range of behavior visitors exhibit within them is even more complex. Very few people have had any formal training in web site analysis, and what training does exist is rudimentary at best. The field of web analytics is in its infancy; standards barely exist, and established principles and indicators of best practice are perhaps a generation away. Most people involved with analyzing their site’s performance therefore lack any idea of how they should be doing. When confronted with a set of numbers they don’t know what those numbers mean – are they indications that things are going well or do they mean the site is barely scrapping along? When it comes to setting targets this same confusion makes it difficult to know what targets you should set, or what is involved in gaining an improvement. For example, if your abandonment rate on your shopping cart is 40%, is getting it to 35% going to be a small step or an astronomical challenge?

People who want benchmarks believe that having the figures from other sites would provide the guidelines to assist in understanding their own site’s behavior and in the setting of reasonable targets. Unfortunately, these people are going to be disappointed.

Online Benchmarks

Benchmarking in the sense of comparing with best practice doesn’t exist in e-commerce. No one is willing to reveal the key performance indicators for their site. Companies don’t reveal their conversion ratios, bounce rates or sales figures. This is because they believe doing so would be to provide direct ammunition to their competitors. Websites are different from other business processes in that they are completely visible to the whole world. Not only can a competitor see your web pages, they can see every aspect of the design in minute detail. Other aspects of a business are harder for an outsider to see, and can never be so transparent. Anyone can see every aspect of how your website takes an order, but they can’t get the same level of understanding of how your office staff do it. The fear is that if a competitor knew the abandonment rate for your shopping cart, they could analyze that page, learn what is working, and emulate this in their site. It is debatable whether this fear is justified, however it is clear that no one wants to reveal their website performance to the rest of the world.

In view of this people are left with the option of looking for aggregated metrics from the online community in general. The hope is that they can compare their site performance with overall averages. Even here numbers are extremely hard to find. IAB sometimes offers some figures, such as their recent analysis of abandonment rates (http://www.iab.net/resources/pdf/DMIDropeview.pdf), but such releases are ad hoc and don’t provide consistency. To my knowledge there is only one consistent source for online benchmarks, the Fireclick Index (http://index.fireclick.com/fireindex.php). However, an examination of the numbers provided by Fireclick (or IAB) reveals little hard information. There are well over 100 million websites in the world. We have no way of knowing what percentage of these sites are commercial. Furthermore we have no way of knowing how many of the commercial sites sell goods online. Even where they do sell goods online, not all accept online credit card payment, or have the same range of payment options. So when IAB or Fireclick offer a single set of top-line numbers for the whole world, it is impossible to know what those numbers mean.

The Fireclick Index is compiled from users of Fireclick’s online web metrics system. Fireclick users are not evenly distributed throughout all industries. For example, Fireclick state they have no users in the travel sector or in online DVD rentals, yet travel is one of the most profitable online sector and DVD rentals is the fastest growing (and extremely profitable). The top line numbers in the Fireclick Index are therefore far from representative. It is also hard to know how the numbers are created. This week they have cited a top line conversion rate of 2.2%. Fireclick break their top-line analysis into six online sectors; fashion, electronics, sports, catalog, specialty and software. These sectors have conversion rates ranging from 0.5% (for both fashion and sports) to 7.1% (for catalog). If you simply take all the sector numbers and average them, the average is 2.6%, not 2.2%. It is obvious therefore Fireclick do not have equal numbers of users in each sector, but unless we know how many in each, it is impossible to know what any of these numbers mean. There are other aspects of the index which are similarly vague, most importantly what constitutes a “conversion”? Finally, some of the numbers seem a little odd. For example, all sectors have shopping cart abandonment rates well over 50%, some as high as 80%. While I have seen some sites with levels getting near that, I doubt 80% is an average abandonment rate for any online sector. It suggests Fireclick’s data is coming from a small sample of poorly designed sites.

No benchmarks

Fireclick’s benchmarks show us why there are no benchmarks for online activity – the range of permutations online business can take is simply too broad. What constitutes a conversion rate? Some companies use visitors per sale, some use unique visitors per sale and some just use visits per sale. The concept of global benchmarks for online performance is even more difficult. Is it reasonable to compare amazon.com with hotels.com? Or to compare either of them with a small home business site. Benchmarks were originally designed to compare things which are very similar in different businesses. Websites may look similar on first glance, but they are remarkably diverse in the way they seek to market and sell to visitors. In many ways the businesses behind the sites show less diversity than the sites themselves.

The desire for benchmarks is born out of a desire to assess performance and set targets. The solution is not to look at others, but to look at your own performance. It doesn’t matter what others are doing. What matters is that you are doing better than before. Trends are more important than single numbers. You should be comparing your performance with previous performance, especially the same time period in previous years (which allows for seasonal variations). An analysis of how much and how quickly numbers have changed in your own site will tell you what targets you can set for improvement, and how long this should take.

Benchmarks for online performance don’t exist and wouldn’t be of value if they did. It doesn’t matter how your competitors are performing. What matters is that your own performance is improving.