The 80/20 Rule

The Pareto Principle

The 80/20 rule, or the Pareto Principle asserts that 20% of something is always responsible for 80% of the results. This is a rule that can be applied to almost anything, and if you apply it appropriately, it can save you from investing time and effort in areas that give you diminishing returns. I mention it here because you can apply the 80/20 rule very effectively to software development, and I use it every day to guide my decisions on where to focus my effort and how to approach project work.

Requirements

So why don't we always just invest 20% of the effort to get 80% of the benefit? In my experience, if you ask people what they want from a given system, they will always ask for everything. They will spend weeks or months coming up with the most comprehensive list of requirements you could imagine. They will, as usual, try to tell you how to implement the system too.

Prioritisation

You can deal with the sheer mass of requirements by converting them first to high level outcomes. These outcomes can be prioritised. Then comes a period of negotiation, where you try and agree on the most appropriate way to deliver each outcome. Once this is done, you can create high level stories that you can estimate. Telling someone that feature X will cost £100,000 soon makes them focus on what is really important. You can then use this additional information to further prioritise these high level stories and see what you can produce for a sensible amount of money, in an iterative manner, achieving real benefits within an acceptable timeframe.

The Malignant Requirement

So we're doing pretty well - we have got a prioritised list of the best value stories that will deliver the outcomes that the business want. We've made sure that the solutions are appropriate, and we understand the impact on the existing infrastructure. Now let's implement that 20% to gain the 80% benefit and move on. But we can't. Why not? Because there is always one Malignant Requirement that the customer will not let go of. This will usually be some really strange, specific requirement, based on some historical pain that they have suffered as a result of an old system or process. Or it could be the sort of requirement that seems really important if you've spent weeks over-analysing the problem domain in isolation.

An Example

They might have had a lot of problems with timeouts in the past because the old system had really long, synchronous call chains. They worked out that it cost the company £500,000 over the past few years, so they are quite happy to spend a huge amount of money on a requirement to dynamically set all timeouts across all technologies and consume real-time data; monitoring each call with respect to the timeout it has been given. The new system has short call chains and an asynchronous architecture, but this isn't taken into account. The fact that dynamic timeouts and real-time monitoring and alerting will make the system far more complex and more expensive to maintain isn't seen either. This one requirement will mean that you put in 80% of the effort, and see 20% of the benefit.

How to Supress the Malignant Requirement

This is a tough one. Trying to educate the customer in the technical considerations is one approach, but this won't work unless they are technical in the first place. Risk and cost-benefit analysis is probably the way to go, and it really helps if you can get an idea of the impact that it will have on future development and maintainance costs. If it will cost £250,000 more over the next 5 years, then this will offset the benefit of the requirement. However you do it, it is really worthwhile trying to supress this requirement, or you lose your ability to capitalise on the 80/20 rule.

Achieving 80/20

Most problems have been solved many times before. You're not the first person to want to monitor the behaviour of your live machines. You're not the first person to want to take orders over the web. These problems have been solved over and over. People write frameworks for this sort of thing, so if you can't find what you need in the opensource community or from a vendor, you need to question your requirements. If your problem domain is really specialised, and the solution that you are looking at really is the first time the industry has encountered this problem, then go ahead and spend your time and resources on solving it. However, if you are writing your own framework for logging, or for website navigation, then question what you're doing. Research on the web. Google, newsgroups, technology sites, find out what's out there. Top technical minds will have spent years solving the problem that you've encountered. They will have learned from previous releases and incorporated this learning into the current release. Bugs will have been fixed, performance optimised. Do you really want to spend years reproducing all this work?

Don't Bespoke the Solution

Another critical point to understand is that if the solution solves 80% of your problems, and its only taken 20% of the effort to implement it, don't go and bespoke the system to try and achieve the remaining 20%. It might seem to make sense to begin with, but it will really hurt you in the longer term. By using a product or solution developed and maintained by a third party, you should be able to benefit from upgrades and bugfixes really cheaply. You might also be able to use third party tools and utilities that integrate with this product to give you instant benefits. If you go for this final 20% to fulfil the Malignant Requirement, you will lose a significant proportion of the 80% benefit you had in the first place. I see it time and time again, where a company has totally bespoked a third party product and then has to maintain and develop it itself, and are unable to upgrade and subsequently fall out of support. The development and maintenance costs are huge, and the system has to be eventually ditched and re-written, incurring further costs.

Conclusion

You can apply the 80/20 rule to all the areas of technology where the problem has been encountered before, enabling you to reuse the solutions that are already out there. The benefit of the snowballs when you look at the costs future development, maintenance and system integration. You should seriously question any requirements that aren't satisfied by a mature, industry-accepted solution. This frees up resource to focus on those areas that are specific to your business, your particular problem domain, and pushing your technology forwards to better serve the business.