Costs of Scaling

Jeff Atwood wrote about scaling applications in a small to medium enterprise environment. I think that Jeff’s numbers are incomplete. However, the commentary on this post does a really good job of filling in the details.

Here are my thoughts on the topic:

Scaling is about software. These days, virtualization is all the rage for scaling. It comes with it’s own share of issues, but it combines the scaling up and out alternatives in such a way that it becomes a question of licensing. By allowing you to add low cost hardware and combine them into larger virtual hardware, it can give you the best of both worlds. It drives the question from how big does the hardware need to be to questions of how quickly one should add additional hardware to the cluster. Or, what is the cost of scaling my software licenses? Does the software I am using have per cpu/core licensing, concurrent licensing, etc? The software, in most of these cases, become the most expensive factor. Energy use and network bandwidth cost typically run a close second and need to be considered. However, for technical decisions in the mid market, software drives hardware decisions.

Whether you are virtualized or not, the decision process does a good job of rendering judgement on the original application architecture. Mid to small tier companies typically have a relatively limited IT budget. While they should be spending upwards of 3%, they very rarely understand the requirements, needs, or potential gains of the investment. As a result, their early technology hiring and architecture decisions are bound by constrained budgets. While, these constraints are necessary for business, sometimes they lead to bad architecture choices based on perceived value and not long term cost effectiveness.

Typically, decisions that are based on these budgetary constraints lead to investments in things like a Microsoft stack or other low cost entry level systems. For the companies that never reach the inflection point that requires a significant growth on their IT infrastructure, these systems do quite well. In fact, they are a good bet. They require few resources to administer, are reliable enough, and fulfill all of the near term business requirements in a nice, neat little package.

The catch is for the companies that do hit the inflection point. The licensing models employed by most large software companies like Microsoft, Oracle, IBM, etc, are built around getting the mid tier companies on their platform at relatively low initial costs. Their business model bets on the idea a certain percentage of those companies will hit an exponential growth curve. This curve will require an expansion of their IT infrastructure which will put them into a different price class. Through a combination of special features, training/hiring investment, and platform lock in, these companies make it less attractive to move to a competitor than to pay the high licensing fees required by the expansion. It is a tightrope that they walk well. In short, they lock the companies in early with entry level deals, only to rape them later.

All of this matters when considering your initial software architecture choices. There are a number of other things that matter. However, cost, short and long term, is king. The art in these decisions is in delineating a long term vision, choosing software that meets that vision, and in choosing software that scales well from a technical and a cost perspective.

My preference is for open source software, which is based on standards, and has an associated support company with enough longevity to be engaged when necessary. That is not the only answer. Microsoft, Oracle, etc solutions can be chosen in such a way to minimize long term costs as well. However, you need the foresight and experience with their licensing models to project costs for future growth. This is difficult in changing economic climates. With open source, I have a few more alternatives to choose from when the plan deviates from the vision. It gives me more options for system integrations and is typically built around open standards. This is why I lean toward open source and standards based solutions. This is also why I hate products that create vendor lock in. I want to be in a position to make the right choices for my clients based on need and future need. I don’t like to make those choices based on the constraints of past mistakes.

This all brings me back to my point – Scaling. A plan for scaling your architecture needs to be built into your initial design. That plan needs to include the costs of scaling your software. Hardware is a commodity. So, when considering how to scale, look to your software choices and the interaction between those software systems. The less friction in these interactions and the more standards based solutions you choose, the more choices you will have when you have to scale. If you have done this well, then you can reduce your scaling costs dramatically over a course of years.