Scalability and performance – Its all about the customers

Todd pinged me to see how I felt about antirez’s suggession in his post titled “On the webserver scalability and speed are almost the same thing“. While I disagree with parts of the post, I can understand why he believed what he wrote.

If someone were to design a state of the art scalable webserver in place of an existing service (lets say apache) which can deliver content in 50 ms, then by my definition the new webserver should continue to serve that content at 50 ms even if the number of requests handled per second by the service increases 100 times. Antirez’s argument is somewhat correct that just because you can scale doesn’t mean that customers will always forgive you for slower performance. But as it happens I’ve seen many different solutions in the last decade which ignored that concern and provided  a scalable yet slower service and still succeeded in generating enough revenue to prove that slower speed may not be a show stopper.

In my opinion its all about the customers. Lets take S3/SimpleDB for example and compare it with mysql and take hosting content in the cloud (SAAS) vs hosting it internally on a corporate web server.  Apart from being able to scale, the reason why customers are ok with using a slower service has to do with two important things which impacts them.

  1. Customer expectations
  2. Customer perception.

If the customer expected responses from S3/SimpleDB should be as fast as Mysql, then the service wouldn’t ever have succeeded. Yet we have Amazon minting money. Whats worse is that in most of these cases the customers were actually developers themselves who had their own set of customers. And most of these customers didn’t have any idea that their service provider was using S3/SimpleDB behind the scenes which is much slower than Mysql or oracle. If they had  noticed their service taking 3 times as long to load, that would have been a huge problem ( and actually a lot of customers do notice when service providers move to the cloud). And this is what most developers complain about… the customer expectations are really high.

Some service providers are pretty good at explaining what customers will gain by accepting slower speed. For example automated backups, auto-upgrades, cheaper hardware might all be good enough reason for “Customer expectation” to be lowered. They might be willing for a 100ms load time if they are compensated in some way by cheaper service.

Another way to solve this is by trying to meet expectations by changing “customer perception”. If you haven’t heard talks by  web UI specialists talk  about how important the “perception of speed” is in the web business, then you should find a talk to attend at one of the conferences. There are tons of ways to speed up serivices… some by complex caching/geo-loadbalancing/compression magic, others by making slight changes to UI to make it look like things are happening fast. For example, a badly designed html page which requires all its images to load before the page is rendered is going to be noticed a lot faster than than one which lots and renders text first and renders images later. Making AJAX calls behind the scenes, preloading data users might need, etc are other forms of such optimizations which can change user perception. Designing your app meet the customer perception is hard but may not be impossible.

Where I disagree with Antirez’s arguments is statement that scalability and performance are “almost” the same thing. They are extremely different because a system which scales doesn’t have to speed up at all… in fact most customers would be happy if 50 ms response time doesn’t change over years even if the number of users has increased 1000 times.

Similar scalability issues exist in every part of our lives. Even you closes grocery chain has this problem. I will be happy with the speed at which they bill me today even if they grew 100 times over the next few years.

BTW I had briefly covered this topic a few years ago in a post called “What is scalabilty“. Feel free to read that as well if you are interested.