Yawn. Getting pretty tired of the programmers and media pundits telling you that scalability should be your focus. Or maybe it’s what people say when they don’t have anything smarter to ask.
Scalability is not a property like size, shape, or lines of code. Scalability is an abstract idea that needs to be applied to a situation continuously. There is no such thing as a scalable solution. You only get a solution that can scale in a given situation. Does your situation scale in the same ways as the software was designed? That’s a really hard question to answer until you have an actual problem to talk about.
Once you have server logs measuring traffic, bandwidth, response times, data sizes, peaks and valleys, and all the rest, you can understand what your particular scaling pattern is.
It’s tempting to generalize and imagine that you’ll need to scale for “high traffic” or “high bandwidth”, but that’s seldom the case. You would be wasting your money to invest in generalized scalability before you have a specific solution.
Here’s another way to think about it: I’ve rarely built a web site that needed more than one or two servers to handle all the visitor traffic. Instead, we needed more servers for scheduled maintenance tasks, like importing data or sending masses of emails. I never know which direction a solution needs to scale, so I stay flexible as long as I can.
3 responses so far ↓
1 Joey
You obviously don’t have a firm grasp on scalability. Scalability encompasses more than than IT architecture. It includes database design and modeling, code framework and efficiency, optimal code refactoring, deployability, and many more factors. They all impact the ability for a site to scale with little work in the future when and if it is needed. It is VERY important for any site that intends on being successful.
2 Ben
I let Joey’s response through moderation to prove my point. He didn’t read very far into my site, or he would have found that we agree on the existence of multifaceted scalability. He provides a great summary. I totally agree.
I disagree with Joey in one huge way: doing things now for a possible payoff “when and if” (as he says) is a really, really bad idea. We disagree on timing, emphasis, and the link between scalability and your success.
Almost every developer you talk to is going to have Joey’s attitude: plan early, architect the shit out of it, build for scalability from day 1, and beat the scalability drum loudly as you parade around spewing theoretical knowledge.
You can go that way. There are many developers who will sell that distraction and burn your budget. But intuitively, you know his viewpoint is wrong. You know from your own life experience that the best laid plans go to waste. The allure of the silver bullet is the only thing that makes Joey’s attitude seem so compelling. We want to believe. It would be so dang convenient and enabling if we could find a way to make big plans that work.
What I am about – and what my philosophy is about – is to know and understand the theory while recognizing that reality throws theory out the window. It doesn’t sound very sexy until you’ve gotten burned by the advice of a few Joeys. After you experience bad advice first hand, you become a believer in Benkido.
Scalability is a form of optimization. You can’t optimize for a general case. You need a concrete situation to study and solve. What’s funny is nearly every programmer will agree that optimizing too early (i.e., before you have a specific problem) is a bad idea. But here’s Joey, seemingly knowledgeable, telling you to do exactly that. And he’s got a lot of company.
Consider this: more projects fail from lack of business analysis than all other reasons combined. I have witnessed and participated in the design of scalable solutions that never ended up launching because they missed the market window, or missed a critical feature, or just plain ran out of money before they could complete the minimum feature set. Everyone focused on scalability and perfect design. We nailed it. And we lost, because our attention and resources were in the wrong place.
Here’s the pragmatic question for you: if scalability doesn’t make you a winner, what does? The answer, of course, is pragmatism. If you focus on the pragmatic choice at each step, you will maximize your chances of success.
For some reason, people really don’t like the idea of not planning ahead very far. I know I sure didn’t like it at first either. It was uncomfortable. Counter-intuitive. I wanted to plan. I needed to plan. But after I let the process work, I saw that pragmatism was leading to ingenious solutions I never would have imagined. Pragmatism is smarter than we are.
My philosophy says that scalability is a by-product of short development iterations and staying tuned in to customer demands. You can’t do that if you spend your whole budget, alone in the woodshed, building a scalable solution to a problem that nobody actually has. The real world is very subtle. Little differences between what you think will work and what actually works carry huge design implications. If you stay focused on what people need – real people who have a problem worth paying money to solve right now, today – your system will evolve into something more beautiful, more scalable, and wildly different than anything you had imagined. Just let evolution take its course. Your software will adapt in very clever ways.
Sorry Joey. I’ve seen it happen. I’ve made it happen. I know what I’m talking about.
3 Ryan Glasgow
Great post and response Ben, I’m glad looking forward to working with you.
Leave a Comment