Traffic statistics and numbers are fun. If you haven’t noticed, we like to post our stats on our blogs and places like Hacker News. However, I have noticed a trend in comments, and that is that people use our numbers and compare them to numbers of some other site using a different stack (often some variation on the LAMP stack).
This just doesn’t make sense. There is really no way to compare the best case performance of two different stacks such as comparing the Microsoft stack to the LAMP stack for dynamic applications. To illustrate this, let us think about what might be required in an experiment that might compare these two stacks in a way that might actually be meaningful.
Things we would need:
- Two teams of programmers and system administrators. You would have a different team for each stack, and the teams would need to be made up of experts on their stack.
- A highly detailed spec about an application that is reasonably feature rich. Each team would need to develop this application on their stack for probably at least a year.
- Enough users using both versions of the application such that people would consider this to be pretty high traffic site over a long period of time. You also would need enough users that possible differences in user behavior would no longer be significant.
Getting the above is never really going to happen, and even if did the technologies probably will have changed by the time the results would be published. Also, even with the things I listed the experiment is pretty weak without many repetitions of the experiment with different teams.
So, why do we post are numbers and do they even mean anything? Well, like I said, they are just fun. However, there is some meaning from our example. Our example shows that it is possible to use the Microsoft stack to do something similar to what we do with the amount of traffic we get with a good team. The scope of this conclusion is limited, but it is also potentially useful to people which is one reason we like to be open about our numbers and our operations.