Monday, June 26, 2006

Agile - Seeing the Light

Agile with Compromises

Until last week, I thought that we'd implemented the Agile process pretty well given the organisational constraints we have. OK, we had to make the odd compromise from the Kent Beck style vision - but we were doing TDD, pair programming, acceptance tests, stories, iterations etc. We were delivering projects and it seemed like a success.

Seeing the Light

A week away as a group of 4 developers and a tester. A room with a handful of dev PC's that are shared. A fresh environment set up with everything configured in the same way. A whiteboard or two.

The customer says what they want. I write this as a story on the whiteboard. We agree on the tasks. We pair up and take the tasks on in order, putting our names by the tasks as we take them on. We switch pairs every few hours. We talk all the time. There are no interuptions. We get through a huge amount of work, and we all know the system at the end. We document the significant details online as we code. We enjoy it, there's free stuff to keep us entertained, we have specialists to ask when we need to get something resolved. We complete the story in a couple of days... That was easy! Why is it so tough to get anything done at the office?

Compromising Agile Principles Impacts Productivity

Working in a fairly standard, open-plan office environment with typical constraints imposed by the organisation appears to adversely affect our productivity. One factor I have identified for this is that other considerations have led us to compromise on many of our Agile practices.

  • We have compromised quite heavily on a single customer always being available to the dev team. We have many customers for each project, and they are rarely available.

  • We have lost some of our confidence to refactor our codebase, as the business perceives this as an unnecessary risk.

  • Architecture, Security and Operations aren't involved on a day-to-day basis with the project team, so can't get their requriements fulfiled as part of the project. This means that they either do a Big Design Up Front, or play the part of Guardian of the Live Environments, stopping projects that don't fit their strategy. They are given a notional say in the project as 'customers', but aren't generally available to be part of the project team.

  • In XP/Agile, no functionality should be added early. However, there are departments that have long-term plans that try and get their solutions implemented a step at a time, but without delivering anything tangible until towards the end of the development effort.

  • Automated acceptance tests have been done, but can't keep up with new developments due to a lack of skills in the testing tool.

  • Moving people around: The company has been structured so that there is little scope to move people around. Moving people around various teams and areas of the system helps to avoid knowledge loss and coding bottle necks. Even if people have the skills to cross the artificial boundaries, they are prevented from doing so by departmental constraints.

  • Release planning is very complicated due to slots available for testing, business delivery deadlines, integration concerns. Therefore we usually have a drop-dead date to hit, and usually with a set deliverable. Not at all Agile.

  • Solutions and timescales are coming through the Propositions process before any dev team has visibility of the project. This restricts the flexibility that the team has to deliver solutions in an Agile way.

  • The tools we use to record stories and tests make it a significant effort to write stories, tasks and iterations, move things around, assign things to people and keep everything up-to-date. Things tend to get out of date due to this overhead.

  • Effective communication: We are communicating using documentation far too much. Documents take a lot of effort to write and keep up-to-date and as a result are usually out-of-date. Far richer communication can take place person-to-person.

The Office Environment Impacts Productivity

Look at what you do every day in the office. How much of it is actually delivering value to the business, and how much is just noise? Its quite enlightening when you look at a breakdown of your day and see how much of your time and effort just gets dispersed into the company without anything to show for it. The Lean Software Development approach is to cut out as much of this waste as possible and concentrate your efforts on delivering value. There are always 'good' reasons to take on the tasks that don't add value, but it doesn't take many of these before they start putting real barriers in the way of doing your job. It doesn't take that long to fill in this form, to go to that meeting. Just a quick chat here, a quick chat there. A firewall preventing you accessing your dev environment, an urgent bug to fix, a problem to head off, a problem with a tool you are using, raising a call with IT support. Suddenly, 80% of your effort has been dispersed and you only have 20% left for the project. The Lean philosophy is to remove as much of this waste as possible.

Team Selection Impacts Productivity

You need to get the right team together to work well in an Agile way. They all need to be very good developers who will write quality code and communicate well. You need to know the skill set of the people that work in the team and ensure that you've got decent coverage of the areas that the project will need to develop for the project. There will be an overlap between the skills, but you should make sure that at least one person in the pair is skilled in the right areas for that particular story. If there is any specialist knowledge required that isn't held within the team, then bring that specialist in to work with the team for a while.

Agile Without Compromises

In our training lab we had the right team of people, the right office environment, the right development environment and a no-compromises Agile process. With all these elements in place, it just worked. It was easy, it seemed the obvious way of working, and nobody was constantly questioning the process. The goal was clear and well-defined by the customer. There were no barriers between the project team and their goal. We were motivated and our workrate increased accordingly. I realised that, in gradually allowing certain elements of the Agile process to be eroded by making compromises, the value of it was severely diminished. These processes and practices support each other and work together to allow the rapid development of simple, quality solutions that deliver business value, quickly. If you start meddling with any part of it, you will expose a weakness with another. So how do we do Agile/XP/Lean Software Development properly, and get the full value?

One Team

It is important that departments such as Security, Architecture and Operations are involved in the development of the solution. However, this involvement needs to be focussed and well managed. They should be part of the project team, with an interest in the project being a success, rather than taking on the role of customers or gatekeepers. They are probably best utilised as specialist consultants, who have all the knowledge of their area, and they bring this to the projects rather than publishing documents and constraining projects.

Good Communication

Agile is about people communicating directly in preference to through documentation. The reason for this is you get for better 'broadband' communication person-to-person over the 'narrow-band' communication through documents. Don't take this as 'Agile projects don't need documentation'. They do. Just document the important stuff, communicate the rest person-to-person.

No Interruptions

If a team can get together and concentrate on a project without interruptions, that will increase productivity massively. When somebody gets interrupted, its not only the time required to deal with the query that is the problem, its the constant context-switching and breaking concentration that has the biggest impact. Open-plan offices can be great, but the ideal for a project team is a room to themselves.

Good Development Environment

Having good, robust environments available, easily refreshed and with shared, high-spec dev PC's is essential. Developer access to all dev boxes is also important. Every time you hit a firewall or permissions problem, is saps time, energy and enthusiasm. You need access to your dev environments.


There are many parts of Agile that are just good programming practices that can be utilised in any process, for example Unit Testing, Continuous Integration and Automated User Acceptance Testing. However, to derive the full business value, quality, speed and flexibility of Agile and XP, you need everything to work together with the right people in the right environment, with a minimum number of constraints and barriers between them and their goal.