Monday, October 30, 2006

Accepting Change


Agile methodologies allow you to accept change at a far greater frequency than Waterfall-based methodologies. However, this does not mean that you can accept constant changes throughout development without it having a detrimental affect on the team's ability to deliver. Most change will cost the project time and money, it will just cost you a lot less in Agile than in Waterfall.

Types of Change

You have to recognise that there are different levels of change:-

  • Incremental

    • First order change

    • Ongoing

    • Evolving

    • Cumulative

    • Continuous

  • Radical

    • Second order change

    • Infrequent

    • Discontinuous

    • Intentional

    • Episodic

These different levels of change can be plotted against how fundamental what you are changing is to the current development. Are you changing something that is core to the implementation (adding new data fields) or something peripheral (some static text).

Accepting Incremental Change

The quadrants illustrate the likely impact of different levels of change. This states that any incremental changes to non-core functionality can easily be accepted during development. So the requirements can evolve during a story, as long as the changes are incremental, cumulative and don’t fundamentally change the structure or direction of the development. These changes will typically come out of discussions between the developers, QAs and the Truth and be recorded as an additional acceptance test.

I see this situation as analogous to that of the die cutter in the Toyota Production System (TPS). The car designers broadly know what shape the body panel is going to be that the die will press. Therefore the die cutter can order the block of material and begin cutting before all the detail is known. As long as the car designers only make incremental and cumulative changes to the panel design, all the initial work will still be valid and nothing will have to be thrown away. However, if they make a fundamental, core change to the shape of the panel, the block of material will have to be thrown away, and the cost of materials, labour and delivery date will be high.

Reacting to Radical Change

If there is a radical change to something core to the implementation, then this will cause a high disturbance and risk to the story under development. These sort of changes are where the Truth says to the developers, mid-story: "I've got a change of context – we actually want it to do this now." Or an interaction designer says "I've changed the page to work like this".

The question is: how do you proceed at this point? You could try and be flexible and change the direction of the implementation and take a bit longer than estimated. However, from experience this approach is damaging to the project. I think that at this point, you have to either stop work on the story and back it out (stop the production line) or continue with the previous requirements until the story is completed and all the acceptance tests are done. The latter option is particularly demoralizing for developers and QAs, spending days writing and acceptance testing something that is going to be ditched. I would therefore favour the former approach – back everything out, delete the code, record the tasks and storypoints that were completed and start with a new story. It seems drastic – particularly if the new requirements aren't completely different, but its usually quicker and cleaner to start again anyway.

Making a Call on Marginal Change

If you get any changes to the Core functionality mid-story, it should probably be rejected at least until you’ve backed out or completed the story in question.
If you get any changes to Peripheral functionality, it's a closer call. If it's a complete (ie Radical) change to the peripheral functionality, then it should be rejected until later in the iteration. Anything less, then look at the impact and make the call – break it out if at all possible, try not to absorb it in the current tasks and estimates.

Sunday, October 29, 2006

Web Application Production Line

Agile User Interface Design

Building a web application following Agile principles has many advantages over other approaches, but there is the challenge of how to integrate the work of graphic designers, interaction designers and web specialist developers into the rapid, iterative cycle.

Web Designers

We tried several approaches to integrating the work of the web designers, who specialize in XHTML, CSS, JavaScript & Flash, into our Agile process. The first approach that we attempted was to pair them with the ASP.NET and C Sharp developers who were producing the rest of the system, but only on the UI focused work. This approach didn't work well, TDD and pair programming didn't add any value, and everyone became frustrated because they couldn't get on with the work they wanted to. Taking this feedback on board, we were able to develop a more effective way of working. The ASP.NET developers would develop totally cut-down XHTML pages, producing only what is absolutely required to make the page work functionally. This page can then be acceptance tested and completed, even though it looks vary basic. This page can then be handed over to the web developers to add the rest of the static content and write the CSS to style the page. As long as the functionality and dynamic content of the page doesn't vary between these two stages, this approach appears to work very well. Its feels slightly Waterfall in its sequentialism, and the throw-it-over-the-fence approach, but we have to recognise that there is a big difference between developing ASP.NET/C Sharp code and producing front-end designs.

Interaction Designers

Traditionally, where I work, the old process was as follows:

  1. Idea is proposed

  2. Interaction Design produce page designs

  3. Graphic designers create all the required images and mock-ups

  4. Customer testing performed using static page mock-ups

  5. Pages get signed off by various departments, including legal

  6. A signed off spec of the screens and copy is handed over to the web designers and developers to implement.

This is very waterfall, and by the time the developers first see the designs, it is already too late to influence them. When the first screens are produced from the early iterations, the feedback loop that is so important to Agile projects is not taken advantage of, because the spec is cast in stone. What we need to do is make this a more iterative, lightweight inclusive process and not a heavy-weight, inflexible, exclusive process.

Learning from the early mistakes we made when we tried to totally integrate the front-end developers into the overall process, I think that we need to recognise that this creative process has to work as something of a production line, but with iterative cycles to get valuable input and feedback into the system as quickly as possible.

In this Web Application Production Line, we need to make sure that every specialist skill is allowed freedom to work to its fullest without being constrained by other parts of the process. We also need to make sure that all the information is included in the feedback loop so that early decisions made with partial information don't constrain the decisions down the line.

Web Application Production Line

So, how to approach this? The initial idea will come from the business or marketing, or an R&D department. If this idea is too concrete with regard to the final implementation, the rest of the process will be constrained and won't work optimally. Therefore we should limit the expression of these ideas to Outcomes, Constraints and High-Level Stories. This gives the development team the freedom to do their jobs effectively.

The next problem is: what skillset is most appropriate for step 2 of the production line? I think that this is Interaction Design. However, currently, Interaction design involves a lot of Graphic Design and static pictures of page designs. I don't think that this is appropriate at this stage of the process. Its too early in the process to be finalising look-and-feel, graphics, spending time getting details just right on pictures produced in Fireworks or some other art package. We need wire-frames at this stage - no fancy graphics, just black and white wire-frames or pencil sketches that capture the fundamental user interactions with the system. We don't want to spend time getting sign-off from all the interested parties, we want the first cut of the user-interactions defined quickly by the people who are skilled in this area. Then they should communicate and discuss their ideas in person as we move on to the next stage of the process, with the expectation that these Interaction Designs will grow and evolve and be changed as part of the iterative process, incorporating feedback from customers and developers.

One aspect of the signed-off spec that comes out of the Interaction Design step in the current process is that all the page designs and mock-ups have been produced in art packages as if they were for paper designs. Hardly any consideration has been given to the fact that this is a page to be displayed on a browser. Is it fixed-width or a liquid layout? How does it expand with the window? Is the design appropriate for the limitations of HTML and CSS? What about accessibility, load-times, degradation between different capabilities of browsers, different screen sizes? The list goes on and on, but the current spec doesn't address any of these points. Despite this, it still gets signed off and passed to the developers.

In my opinion, the next stage of the process needs the front-end specialist developers (Box 2) (XHTML, CSS with appreciation of UID and Graphic Design) involved to quickly get some basic XHTML documents produced, with maybe some quick (standard?) CSS applied to give the page some structure and the right sort of image. At this point, the browser-specific considerations mentioned above will come into focus and these can be discussed with the interaction designers. However, because the wireframes are so basic, they should not constrain the web designers, and they should have the freedom to augment the early designs with their input and record their decisions in the form of an XHTML document.

This is where you can start your user testing and initial feedback loop. Real customers might not be appropriate, given the rudimentary nature of the pages at this early stage, but within a web development team, there are many people who can give valuable feedback at this point. The Truth (Business Representative on the team), QAs (testers), Interaction Designers and Web Designers are all good candidates to try the early pages and give feedback.

You can go round this initial loop (Box1, Box2) until everybody is happy that the interactions are optimal, the basic layout is acceptable, and the XHTML, accessibility and cross-browser compatibility hit the right level. Then this can be "signed off" by the Interaction and Web Designers as their requirements spec, and put into source control. The code is the documentation. This static HTML could include comments as placeholders for required dynamic content, and all the various XHTML fragments to support all scenarios that can occur in the page so that the ASP.NET (or JSP) developer can easily understand what he needs to do. Everyone should work on this source-controlled XHTML document from this point onwards.

It os at this stage that the Graphic Designers, the Copywriters, Legal, Brand etc can get involved and, using the XHTML document as the master copy of the spec/design, work to build the finished article with the correct Brand, legally compliant text and the correct images.

Design Work with Planning and Stories

A sticking point with this production line and XP Planning is that once an Iteration has been defined, the team immediately begins work on the highest priority stories. However, if you have to wait for the Interaction and Web Designers to finish their stage of the process before you can start coding your ASP.NET page, what do you do? Its obvious that the faster and more efficient you can get the first stage of the process, the less latency you will have in the system. The less time the Interaction Designers spend looking ahead, the more focussed and efficient they can be on the highest priority Story that the team is working on. Therefore cutting down the scope of Interaction Design to purely producing wireframes for the story in question does help. However, if everyone works on the same iteration at the same time, how do you get rid of this lag? One approach that appeared to work quite well is to create the wireframes and XHTML as part of the Planning Game. As the pages are so basic, it shouldn't take more than a few hours, and then this can be used as a contract between the various areas to base their work on. While the ASP.NET and C Sharp developers are creating and estimating their tasks, the Interaction and Web Designers can be putting together the basics of the screen flow. The rest of the work (CSS, JavaScript, Flash, Graphic Design, ASP.NET, Web Service, Database, Copy, Customer Testing) can be based on this hub of an XHTML skeleton. Automated tests to ensure that this XHTML skeleton isn't changed would give an early warning of any problems being created.

The Car Analogy

I always use a car anaolgy, so see the XHTML document as the body-shell of a car at the start of the production line. It picks up its engine, wheels, interior, electrics and paint job as it progresses past all the various stages. As long as everyone works on the same body-shell and nobody changes the structure of the shell as they do their work, then at the end the car will have all the right features and fulfil the requirements of the initial design. However, if anyone does make a fundamental change to the structure of the body shell, then we have to immediately stop the production line (Jidoka) and fix the problem straight away, with all departments involved in the resolution. The cost of this needs to be measured and recorded to ensure that it is understood what caused this waste and avoid it happening again. We have tried to continue in this circumstance, and it always causes big problems later, and nobody is clear what caused the problem in the first place, so repeat the mistake. Its best to either fully complete the work as you had intended initially including full acceptance testing, or totally back out all the changes made to that point. Its a tough call, but the alternative is far more painful.