Economy Size Geek

This is probably one of those things that is already obvious, becomes obvious once you read this post, or you never thought about it :)

I spend a lot of my time building systems. Especially systems to help people get processes done using software. One of the things I had never taken much time to think about is the pressure that growth puts on a system/process.

Bascially there is a huge difference between desigining a system for 100,000 users and one that is designed for 100 users but ends up having 100,000 users. That is not to say that if you knew up front that there were going to be some many people it would be absolutely simple to plan for it (especially if you had never worked on a system before that had so many users). It’s more that knowing up front that it was going to be big makes it easier to justify building in extra support in the system to handle the weight of all those users.

With 100 users you can think about how do I keep this small and simple. The other one you know that performance is going to be sensitive from the start. What I’m really getting at is that sometimes you build a piece of software or you design a work flow and it seems fine - actually it seems great. Then it goes into action and it functions correctly. If you just did it a little more often you might occasiontally find problems with it (there is a dependancy you weren’t aware of in the work flow that can sometimes slow things down) but really those are just the normal bumps in the road. But if every day the process is used more and more because there is an ever escalating number of customers - the problem isn’t just that you find a problem in the flow that needs a solution - but instead that every day there is a new and different problem that needs to be fixed. The pressure of growth adds stress to the system and that stress makes any flaws or weaknesses in the process show themselves that much quicker.

I’ve tended to beat myself up about this sort of thing. Especially because I’m normally very good and making it so there are not a lot points in the process that can buckle, but it still happens. And for a long time I wondered why it seemed like new things came up over time instead of the same thing over and over - and it turns out this is the reason. Every environment I have evern built for starts small with a woefully inadequate estimate of how large it is going to get - or how much it is going to change.

One school of thought is probably that you should just plan more upfront and weigh in the factor that everything is going to grow too much, but that just doesn’t jive with me. My view is to stick to my Agile knitting - and say that the my solution is to try to keep things as simple as possible with the understanding that there will be gaps. There will always be gaps, so instead of beating myself up about it , I want to build things that make it easier to fill the gaps that matter the most. This path may start in chaos but it will quickly stablize if you pay attention and adapt the system to what is going on. I’m sure there are examples where this is a bad idea - but so far it turns out I don’t work on those….