How to Write a User Story

So What is a User Story?

A User Story is used in eXtreme Programming (XP) to define what the customer wants the system to do for them. They replace the traditional requirement specs and use case diagrams.

The Who, the What and the Why

There are many ways to approach writing a User Story. The following approach has worked best for me:-

As [the customer] I want [a capability] so that I can [achieve an outcome]

Its the WHO, the WHAT and the WHY. Who wants this capability, what is it, and why do they want it? It is useful to define the Who and the Why because this makes you consider who wants this feature and what you will achieve with it. If you have difficulty defining either of these, then you probably don't need the feature in the first place.

The Five Whys

To get to the actual, core reason that you require the story, it is sometimes necessary to peel back a few layers. A technique called 'The Five Whys', commonly used to fine root causes of problems, can be useful here. A similar approach can also be applied to finding the actual end-consumer for a story.

Capture the Essence of the Requirement

The Story does not have to, and should not, fully explain every last detail of the requirements. It should be brief, and form a hub for discussion and collaboration. A Story also has acceptance tests and tasks associated with it. The acceptance tests are what fully define the story. To enable you to frame the problem and define the Story to a level where it can be estimated and prioritied, you will probably need a few, high level acceptance tests. You can then get more specific and fine-grained in your tests during development.

Ditch the Jargon

Most Stories should be non-technical, they are capturing a requirement, not the technical implementation of the solution. The alarm bells should go off if there is technical jargon in the Story text. Avoiding describing the solution in the Story also gives you more options, such as solving the problem with a manual process, or a different automated solution that becomes apparent during discussions with developers and testers.

Vertical Slices through the System

It is important that each story represents a slice through the system, eg web page, web service, database. Splitting Stories horizontally, by technical area or layer in the system should be avoided.

Example

As the stock delivery manager, I want to be informed when any stock falls below its threshold level so that I can order more stock in before the store runs out.

Acceptance Tests and Tasks would be added during the Getting the Story Straight and Planning Game sessions.

INVEST in Good Stories, and SMART Tasks
Writing Good User Stories
User Stories

Project Roles

Roles Within Project Teams

For a team to work effectively, they need clearly defined roles, and individuals need to be empowered to fulfil these roles.

Project Manager (PM)

• Budget holder


• Owns overall project plan with timescales, deliverables & outcomes.


• Understands high level project progress and issues


• Provides environment to enable team to deliver


• Shields team from details of the business and politics.


• Deals with resourcing requirements


• Represents the project to senior management


• Works with Development Managers and Dev Leads to pull together the required proposition and take it into delivery if appropriate.

Empowerment Required: Demand that their outcomes are achieved by the dev team.

Team Leader

• People manager with technical background


• Runs sessions (eg standup, retrospective, planning)


• Supports and enforces development process by dealing with
  
  
  
  • people management
  
  
  • planning
  
  
  • progress measurment
  
  
  • external relationships
  
  
  • company politics
  
  
  • communication channels
  
  


• Works closely with dev team - especially Team Leader


• Attends relevant external meetings on behalf of team


• Ensures that the team operates smoothly, and have the tools & resource to deliver the project outcomes.


• Deals with issues that need resolving within and outside the team.


• Understands the technical details more than PM, but not to the depth of the senior developers.

Empowerment Required:

• Represents team


• Authority figure within team


• Runs project day-to-day


• PM primarily interacts with team via Team Leader.

Truth

• Provides the single point of requirements for the development team.


• Is able to answer all questions relating to requirements in a timely manner.


• Is available to the team at all times.

Empowerment Required:

• Authority to answer questions on requirements.


• Authority to make judgement calls on-the-spot.


• Empowered to commit all their time to the project and team.


• One Source for project requirements.

Lead Tester (QA)

• Works closely with Truth and Developers to understand requirements and provide acceptance tests, both manual and automated.


• Runs tests and sign off each story, iteration and release.


• Raises bugs.

Empowerment Required

• The acceptance tests they create are the single source of the requirements.


• If Truth is unavailable within acceptable timeframe, has authority to make the decision.


• Power to sign-off or reject stories, iteration and releases.

Marketing/Business

• Combines business knowledge and market research to generate initial high-level ideas, outcomes and goals.


• Sells the idea to the business and gets budget allocated to programme.


• Understands the long-term plan


• Generates the outcomes that are passed to the project team to fulfil.


• Understands how the outcomes fulfil the business requirements

Empowerment Required: To demand that their outcomes are achieved.

Technical Lead (Coach)

• The most senior developer on the team


• Takes responsibility for technical decisions on the project in conjunction with Architecture and other departments.


• Responsible for the technical output of the team, and ensuring that it complies with both internal and industry standards and best practices.


• Responsible for ensuring that the technical practices that support the process are implemented within the team eg Test Driven Development, Pairing, Continuous Integration.


• Works as a developers, providing coaching in Agile practices and coding.


• Works closely with Team Leader.

Empowerment Required: Empowered to make the technical decisions.
Empowered to use best practices and apply both internal and industry standards.

Developer

• Understands and embraces the Agile process, its benefits and core practices


• Is able to produce good quality, well designed code in response to stories


• Participates well within the team, adopts all required practices and attends all sessions.


• Works closely with Tester (QA) on Story


• Recognises the authority and expertise of Technical Lead (Coach) and uses them appropriately.

Web Application Production Line II

The Problems

Design Spec

User Interface Design use a process that is more appropriate to the Waterfall methodology. A lot of up-front design time is spent finalizing page designs, page flow, graphics and copy, running customer workshops and updating design decisions based on their feedback. Then a final spec is produced, which has to be signed-off by many interested parties, before it is handed over to development to implement.

The output from this stage of the process creates problems for the rest of the process for the following reasons:-

• The spec usually takes so long to be finalised, that the development project is usually nearing completion before it is signed off.


• Late changes to spec negatively impact the project just before final testing


• The spec contains static pictures of page designs, but there aren’t any HTML prototypes, so they haven’t taken into account the limitations or features of HTML and CSS.


• There is very limited scope for iterative feedback and prototyping, and every change will have to go through sign-off again, causing more delays.


• When the developers begin their work on the pages, they learn far more about the problem area, and have feedback that can substantially change the page designs.

Brand Guidelines

Brand guidelines aren't always appropriate for today’s internet. They prescribe layout that doesn’t take into account the limitations and features of XHTML and CSS or different devices. Brand can be more appropriate to print media and needs to be less prescriptive and give guidelines for look-and-feel rather than dictating every pixel on the page.

Current Process

Traditionally, the process is as follows:

• Idea is proposed


• User Interface Design produce full page designs on paper


• Graphic designers create all the required images, mock-ups and Flash demos


• Customer testing performed using static page mock-ups and Flash demos


• Copy is created and sent to Legal for sign-off


• Full spec is created with user flow, static screen designs and copy.


• Spec gets signed off by various departments, including legal and brand


• A signed off spec of the screens and copy is handed over to the front-end developers and C# developers to implement.

This is very waterfall, and by the time the developers first see the designs, it is already too late to influence them. When the first XHTML prototypes are produced from the front-end developers, the feedback loop is not taken advantage of, because the spec is carved in stone. What we need to do is make this a more iterative, lightweight inclusive process and not a heavy-weight, inflexible, exclusive process.

I think that we need to recognise that this creative process has to work as something of a production line, but incorporating increasingly inclusive iterative cycles to gather valuable input and feedback as quickly as possible.

In this Web Application Production Line, we need to make sure that every specialist skill is allowed freedom to work to its fullest without being constrained by other parts of the process. We also need to make sure that all the information is included in the feedback loop so that early decisions made with partial information don't constrain the decisions down the line.

Web Application Production Line

The initial idea will come from the business or marketing, or an R&D department. If this idea is too concrete with regard to the final implementation, the rest of the process will be constrained and won't work optimally. Therefore we should limit the expression of these ideas to Outcomes. This gives the development team the freedom to do their jobs effectively.

The next problem is: what skillset is most appropriate for step 2 of the production line? I think that this is User Interface Design. However, currently, the first stage of our process involves a lot of Graphic Design and static pictures of page designs. I don't think that this is appropriate at this stage of the process. Its too early in the process to be finalising look-and-feel, graphics, spending time getting details just right on pictures produced in Fireworks or some other art package. We need wire-frames and storyboards at this stage - no fancy graphics, just basic sketches that capture the fundamental user interactions with the system, the information displayed or captured, and the scenarios that need to be catered for.

We don't want to spend time getting sign-off from all the interested parties, we want the first cut of the user-interactions defined quickly by the people who are skilled in this area. Then they should communicate and discuss their ideas in person as we move on to the next stage of the process, with the expectation that these wireframes and storyboards will grow and evolve and be changed as part of the iterative process, incorporating feedback from users and developers.

One aspect of the signed-off spec that comes out of the Design step in the current process is that all the page designs and mock-ups have been produced in art packages as if they were for paper designs. Hardly any consideration has been given to the fact that this is a page to be displayed on a browser. Is it fixed-width or a liquid layout? How does it expand with the window? Is the design appropriate for the limitations of HTML and CSS? What about accessibility, load-times, degradation between different capabilities of browsers, different screen sizes? The list goes on and on, but the current spec doesn't address any of these points. Despite this, it still gets signed off and passed to the developers.

In my opinion, the next stage of the process needs the front-end specialist developers (XHTML, CSS with appreciation of UID and Graphic Design) involved to quickly get some basic XHTML documents produced, with maybe some quick (standard?) CSS applied to give the page some structure and the right sort of image. At this point, the browser-specific considerations mentioned above will come into focus and these can be discussed with the interaction designers. However, because the wireframes are so basic, they should not constrain the front-end developers, and they should have the freedom to augment the early designs with their input and record their decisions in the form of an XHTML document.

This is where you can start your user testing and initial feedback loop. Real customers might not be appropriate, given the rudimentary nature of the pages at this early stage, but within a web development team, there are many people who can give valuable feedback at this point. The Truth (Business Representative on the team), QAs (testers), Usability and Front-end Developers are all good candidates to try the early pages and give feedback. Quite often, this unearths some fairly basic scenarios that haven’t been catered for in the design which then needs some more work before it can be developed.

You can go round this initial loop until everybody is happy that the interactions are optimal, the basic layout is acceptable, and the XHTML, accessibility and cross-browser compatibility hit the right level. Then this can be "signed off" by the Design team as their requirements spec, and put into source control. This static XHTML could include comments as placeholders for required dynamic content, and all the various XHTML fragments to support all scenarios that can occur in the page so that the ASP.NET developer can easily understand what he needs to do. Everyone should work on this source-controlled XHTML document from this point onwards.

It is at this stage that the Graphic Designers, the Copywriters, Legal, Brand etc can get involved and, using the XHTML document as the master copy of the spec/design, work to build the finished article with the correct Brand, legally compliant text and the final images.

Design Work in the Planning Phase

A sticking point with this production line and iterative implementation is that once the project requirements have been defined, we want the dev team to begins work on the highest priorities as soon as possible. However, if developers have to wait for the Design stage of the process before they start coding, this will cause a delay, and therefore waste. It's obvious that the faster and more efficient you can get the initial Design stage of the process, the less latency and waste you will have in the system. We first need to ensure that the Design stage is focusing on the pages that the developers need to develop first. It sounds obvious, but it doesn’t happen at the moment. Also, cutting down the scope of the Design stage to purely produce wireframes and storyboards will help speed this up. However, if we want everyone to work on the same pages at the same time, how do you get rid of the lag that the Design phase introduces? One approach that appears to work quite well is to create the wireframes, storyboards and XHTML as part of the Planning stage, possibly in parallel with the preceding iteration. Once the XHTML pages are produced, they can be used as a contract between the various areas to base their work on. While the ASP.NET and C# developers are planning and estimating their work, the Design team can be putting together the basics of the screen flow. The rest of the work (CSS, JavaScript, Flash, Graphic Design, ASP.NET, Web Service, Database, Copy, Legal, Testing) can be based on this hub of an XHTML skeleton.

References

• Wireframes


• Wireframes


• HTML Prototypes


• Storyboards


• Interactive Wireframes

Managing the Team

Give the Team Freedom

I just read an interview with Mike Gascoyne, who is the Chief Technology Officer at an F1 team. Couple of extracts that I thought were relevant to managing Dev Teams:

...part of being a good manager is that you leave your guys free to do their jobs

...no matter how much money you put into building the house, if you don't dig the foundations properly, it falls down. So you've got to do the basics right

...if you go in and start changing everything all that will mean is they won't actually finish it, and everything will be half and half

...the key thing is to put the structures in place, get the priorities set

Accepting Change

Introduction

Agile methodologies allow you to accept change at a far greater frequency than Waterfall-based methodologies. However, this does not mean that you can accept constant changes throughout development without it having a detrimental affect on the team's ability to deliver. Most change will cost the project time and money, it will just cost you a lot less in Agile than in Waterfall.

Types of Change

You have to recognise that there are different levels of change:-

• Incremental




• First order change


• Ongoing


• Evolving


• Cumulative


• Continuous

• Radical




• Second order change


• Infrequent


• Discontinuous


• Intentional


• Episodic

These different levels of change can be plotted against how fundamental what you are changing is to the current development. Are you changing something that is core to the implementation (adding new data fields) or something peripheral (some static text).

Accepting Incremental Change

The quadrants illustrate the likely impact of different levels of change. This states that any incremental changes to non-core functionality can easily be accepted during development. So the requirements can evolve during a story, as long as the changes are incremental, cumulative and don’t fundamentally change the structure or direction of the development. These changes will typically come out of discussions between the developers, QAs and the Truth and be recorded as an additional acceptance test.

I see this situation as analogous to that of the die cutter in the Toyota Production System (TPS). The car designers broadly know what shape the body panel is going to be that the die will press. Therefore the die cutter can order the block of material and begin cutting before all the detail is known. As long as the car designers only make incremental and cumulative changes to the panel design, all the initial work will still be valid and nothing will have to be thrown away. However, if they make a fundamental, core change to the shape of the panel, the block of material will have to be thrown away, and the cost of materials, labour and delivery date will be high.

Reacting to Radical Change

If there is a radical change to something core to the implementation, then this will cause a high disturbance and risk to the story under development. These sort of changes are where the Truth says to the developers, mid-story: "I've got a change of context – we actually want it to do this now." Or an interaction designer says "I've changed the page to work like this".

The question is: how do you proceed at this point? You could try and be flexible and change the direction of the implementation and take a bit longer than estimated. However, from experience this approach is damaging to the project. I think that at this point, you have to either stop work on the story and back it out (stop the production line) or continue with the previous requirements until the story is completed and all the acceptance tests are done. The latter option is particularly demoralizing for developers and QAs, spending days writing and acceptance testing something that is going to be ditched. I would therefore favour the former approach – back everything out, delete the code, record the tasks and storypoints that were completed and start with a new story. It seems drastic – particularly if the new requirements aren't completely different, but its usually quicker and cleaner to start again anyway.

Making a Call on Marginal Change

If you get any changes to the Core functionality mid-story, it should probably be rejected at least until you’ve backed out or completed the story in question.
If you get any changes to Peripheral functionality, it's a closer call. If it's a complete (ie Radical) change to the peripheral functionality, then it should be rejected until later in the iteration. Anything less, then look at the impact and make the call – break it out if at all possible, try not to absorb it in the current tasks and estimates.

Web Application Production Line

Agile User Interface Design

Building a web application following Agile principles has many advantages over other approaches, but there is the challenge of how to integrate the work of graphic designers, interaction designers and web specialist developers into the rapid, iterative cycle.

Web Designers

We tried several approaches to integrating the work of the web designers, who specialize in XHTML, CSS, JavaScript & Flash, into our Agile process. The first approach that we attempted was to pair them with the ASP.NET and C Sharp developers who were producing the rest of the system, but only on the UI focused work. This approach didn't work well, TDD and pair programming didn't add any value, and everyone became frustrated because they couldn't get on with the work they wanted to. Taking this feedback on board, we were able to develop a more effective way of working. The ASP.NET developers would develop totally cut-down XHTML pages, producing only what is absolutely required to make the page work functionally. This page can then be acceptance tested and completed, even though it looks vary basic. This page can then be handed over to the web developers to add the rest of the static content and write the CSS to style the page. As long as the functionality and dynamic content of the page doesn't vary between these two stages, this approach appears to work very well. Its feels slightly Waterfall in its sequentialism, and the throw-it-over-the-fence approach, but we have to recognise that there is a big difference between developing ASP.NET/C Sharp code and producing front-end designs.

Interaction Designers

Traditionally, where I work, the old process was as follows:

1. Idea is proposed


2. Interaction Design produce page designs


3. Graphic designers create all the required images and mock-ups


4. Customer testing performed using static page mock-ups


5. Pages get signed off by various departments, including legal


6. A signed off spec of the screens and copy is handed over to the web designers and developers to implement.

This is very waterfall, and by the time the developers first see the designs, it is already too late to influence them. When the first screens are produced from the early iterations, the feedback loop that is so important to Agile projects is not taken advantage of, because the spec is cast in stone. What we need to do is make this a more iterative, lightweight inclusive process and not a heavy-weight, inflexible, exclusive process.

Learning from the early mistakes we made when we tried to totally integrate the front-end developers into the overall process, I think that we need to recognise that this creative process has to work as something of a production line, but with iterative cycles to get valuable input and feedback into the system as quickly as possible.

In this Web Application Production Line, we need to make sure that every specialist skill is allowed freedom to work to its fullest without being constrained by other parts of the process. We also need to make sure that all the information is included in the feedback loop so that early decisions made with partial information don't constrain the decisions down the line.

Web Application Production Line

So, how to approach this? The initial idea will come from the business or marketing, or an R&D department. If this idea is too concrete with regard to the final implementation, the rest of the process will be constrained and won't work optimally. Therefore we should limit the expression of these ideas to Outcomes, Constraints and High-Level Stories. This gives the development team the freedom to do their jobs effectively.

The next problem is: what skillset is most appropriate for step 2 of the production line? I think that this is Interaction Design. However, currently, Interaction design involves a lot of Graphic Design and static pictures of page designs. I don't think that this is appropriate at this stage of the process. Its too early in the process to be finalising look-and-feel, graphics, spending time getting details just right on pictures produced in Fireworks or some other art package. We need wire-frames at this stage - no fancy graphics, just black and white wire-frames or pencil sketches that capture the fundamental user interactions with the system. We don't want to spend time getting sign-off from all the interested parties, we want the first cut of the user-interactions defined quickly by the people who are skilled in this area. Then they should communicate and discuss their ideas in person as we move on to the next stage of the process, with the expectation that these Interaction Designs will grow and evolve and be changed as part of the iterative process, incorporating feedback from customers and developers.

One aspect of the signed-off spec that comes out of the Interaction Design step in the current process is that all the page designs and mock-ups have been produced in art packages as if they were for paper designs. Hardly any consideration has been given to the fact that this is a page to be displayed on a browser. Is it fixed-width or a liquid layout? How does it expand with the window? Is the design appropriate for the limitations of HTML and CSS? What about accessibility, load-times, degradation between different capabilities of browsers, different screen sizes? The list goes on and on, but the current spec doesn't address any of these points. Despite this, it still gets signed off and passed to the developers.

In my opinion, the next stage of the process needs the front-end specialist developers (Box 2) (XHTML, CSS with appreciation of UID and Graphic Design) involved to quickly get some basic XHTML documents produced, with maybe some quick (standard?) CSS applied to give the page some structure and the right sort of image. At this point, the browser-specific considerations mentioned above will come into focus and these can be discussed with the interaction designers. However, because the wireframes are so basic, they should not constrain the web designers, and they should have the freedom to augment the early designs with their input and record their decisions in the form of an XHTML document.

This is where you can start your user testing and initial feedback loop. Real customers might not be appropriate, given the rudimentary nature of the pages at this early stage, but within a web development team, there are many people who can give valuable feedback at this point. The Truth (Business Representative on the team), QAs (testers), Interaction Designers and Web Designers are all good candidates to try the early pages and give feedback.

You can go round this initial loop (Box1, Box2) until everybody is happy that the interactions are optimal, the basic layout is acceptable, and the XHTML, accessibility and cross-browser compatibility hit the right level. Then this can be "signed off" by the Interaction and Web Designers as their requirements spec, and put into source control. The code is the documentation. This static HTML could include comments as placeholders for required dynamic content, and all the various XHTML fragments to support all scenarios that can occur in the page so that the ASP.NET (or JSP) developer can easily understand what he needs to do. Everyone should work on this source-controlled XHTML document from this point onwards.

It os at this stage that the Graphic Designers, the Copywriters, Legal, Brand etc can get involved and, using the XHTML document as the master copy of the spec/design, work to build the finished article with the correct Brand, legally compliant text and the correct images.

Design Work with Planning and Stories

A sticking point with this production line and XP Planning is that once an Iteration has been defined, the team immediately begins work on the highest priority stories. However, if you have to wait for the Interaction and Web Designers to finish their stage of the process before you can start coding your ASP.NET page, what do you do? Its obvious that the faster and more efficient you can get the first stage of the process, the less latency you will have in the system. The less time the Interaction Designers spend looking ahead, the more focussed and efficient they can be on the highest priority Story that the team is working on. Therefore cutting down the scope of Interaction Design to purely producing wireframes for the story in question does help. However, if everyone works on the same iteration at the same time, how do you get rid of this lag? One approach that appeared to work quite well is to create the wireframes and XHTML as part of the Planning Game. As the pages are so basic, it shouldn't take more than a few hours, and then this can be used as a contract between the various areas to base their work on. While the ASP.NET and C Sharp developers are creating and estimating their tasks, the Interaction and Web Designers can be putting together the basics of the screen flow. The rest of the work (CSS, JavaScript, Flash, Graphic Design, ASP.NET, Web Service, Database, Copy, Customer Testing) can be based on this hub of an XHTML skeleton. Automated tests to ensure that this XHTML skeleton isn't changed would give an early warning of any problems being created.

The Car Analogy

I always use a car anaolgy, so see the XHTML document as the body-shell of a car at the start of the production line. It picks up its engine, wheels, interior, electrics and paint job as it progresses past all the various stages. As long as everyone works on the same body-shell and nobody changes the structure of the shell as they do their work, then at the end the car will have all the right features and fulfil the requirements of the initial design. However, if anyone does make a fundamental change to the structure of the body shell, then we have to immediately stop the production line (Jidoka) and fix the problem straight away, with all departments involved in the resolution. The cost of this needs to be measured and recorded to ensure that it is understood what caused this waste and avoid it happening again. We have tried to continue in this circumstance, and it always causes big problems later, and nobody is clear what caused the problem in the first place, so repeat the mistake. Its best to either fully complete the work as you had intended initially including full acceptance testing, or totally back out all the changes made to that point. Its a tough call, but the alternative is far more painful.

References

• The Toyota Way


• Agile Web Design Manifesto


• Agile UID


• Agile Web Design Case Study


• Agile

The Toyota Way

Introduction

The "Toyota Production System" (TPS) is a Lean approach to manufacturing, pioneered by Taiichi Ohno, that focuses on eliminating waste to make the process of producing a car more efficient. The principles employed by TPS can be applied to many industries, software development being a prime example.

The Principles

Jidoka

The ability to stop production lines.

Just-In-Time

What is needed, when it is needed.

Kaizen

The concept of continuous improvement.

Muda

Non-value added. Muda is translated as waste.

There are many other principles, defined here.

Books

• Toyota Production System: Beyond Large-scale Production - by Taiichi Ohno
• The Toyota Way: 14 Management Principles from the World's Greatest Manufacturer - by Jeffrey Liker


• Lean Software Development: An Agile Toolkit - by Mary Poppendieck
  

Google Gets Today's Internet

Google Get the Internet

Google are delivering on the promise of the modern internet, and it illustrates to everyone else what we are striving to achieve. Google Personalized Home gives you a personalized page full of widgets/gadgets/modules that you can drag and drop. These include RSS feeds, Gmail inbox summary and other mini web apps. You can easily write your own and plug them in. Gmail is a very good email provider, and is accessible via the web and over POP.

Mobile Web

You can use all these features on your mobile phone too. And this isn't like the false dawn of WAP - it really does work. The mobile version of Google personalized Home is easy to set up and use, as is Gmail, and they both work on basic XHTML browsers.

The Rest

Google Earth is amazing, and is a great example of a smart client, using their form of Ajax to give a seamless experience on your desktop whilst asynchronously using web services behind the scenes. Google Maps is a great example of a Rich Internet Application (RIA) making the user experience far more intuitive and effortless. In addition, the site that I'm using for this blog, Blogger is a very good example of an XHTML site.

This is Not an Advert

This may sound like an advert for Google, but I promise you it isn't. Its just that Google obviously get what opportunities the modern internet offers, and they are really delivering on that. Not many people seem to have this understanding, and I spend a lot of my time trying to explain why I'm so focussed on standards complient XHTML with CSS, and on a SOA. Its also why I don't want to fix the bug in the website which is actually a bug in Internet Explorer's standards support. If you just try and build a site that works on all versions of Internet Explorer, you are missing the point. That was the 90's. Please, get with the program - see what you're missing out on.

Lie About Quality

Quality is the Constant

The controls you have over a project are scope, time, resource and quality. XP teaches us never to use quality as a control, it should always be 100%. However, I've recently noticed a strange phenomenon: the business is scared of quality. I try to say that we will deliver a high quality product, and the response is unexpected. That sounds expensive - can't you just throw something together quickly? At this point, I consider whether to talk about maintenance costs and total cost of ownership, but think better of it. Why do the business want a low quality solution? Its because they perceieve this to be cheaper and quicker to develop. But we know that with XP we can produce quality software very quickly. However, this is very difficult to communicate to the business sponsor. What to do?

A Novel Strategy

It seems a strange approach, but I can only see one way to overcome this problem. Lie. Pretend that you are throwing together a solution of questionable quality in order to meet the deadline. Sell it as Rapid Incremental Delivery of a prototype, RAD, JFDI, whatever. However, behind the scenes put in place all the elements of XP: Continuous Integration, TDD, Automated Acceptance Testing, Stories, Iterations, Release Planning, Pair Programming. Just don't tell anyone. They will get their solution rapidly and they will get incremental delivery of features. However, when they change their minds about the fact that this is a temporary solution, and leave it in live for 3 years, you will be confident of its quality and won't get bitten by a hidden maintenance cost. When you are asked to enhance the solution in a few months time, you will be able to do this in confidence, as you will have good test coverage.

Agile Architecture

Architecture doesn't have to suffer either. As long as you map out the architectural components of the system and assign them their responsibilities, data and interfaces, then you can implement them in the most minimal way possible - even to the extent of a manual process. As long as the system boundaries are well defined, you can fulfill the architectural component's responsibilities however you see fit. However, follow the same principle of lying to the business. If a quality system scares them, just wait for the reaction when you tell them you are going to build a system that adheres to architectural guidelines. That sounds very expensive. Can't you just throw something together - we could make it comply to architecture later...? Pretend you are breaking every architectural rule in the book in order to get the solution out quickly and cheaply, but quietly make sure that the architecture is well defined and adhered to. It doesn't take long to create a good architecture and build high quality systems, just don't tell anyone...

Conceptual Thinking

Introduction

Over the past few months, I have noticed that people, and their approach to solving problems, can be broadly split into two groups. The first group adopt a Concept Driven approach, where they first visualise the logical solution to the problem, without constraints. The second group take what they have today as the starting point.

Two Different Approaches

People naturally fall into one of the following approaches when faced with a problem to solve:-

Concept Driven Approach

In response to a problem, this group initially form a high level, conceptual idea of how the solution should work. Then they look at the best way of implementing this model without making too many compromises. Once the logical structure, division of responsibilities, interfaces and patterns are defined, the next step is to see how much reuse can be achieved whilst adhering to this conceptual model.

Linear, Incremental Approach

In response to a problem, this second group look immediately to what they already have available. Once they find something that appears to do something similar, they try to extend or modify it to satisfy the requirements.

Example

To illustrate these differing approaches, I will return to my favorite analogy: building a car. If a designer pens a beautiful concept car, and the car company decides that they need a car like this in their range, there are two approaches they could take. They could build the basics of the new concept, for example the bodyshell, and then use their parts bin to put in a standard, pre-existing drivetrain, engine and suspension. This approach would retain a purity of concept whilst achieving high reuse of proven components. The other approach would be to find the car in their range that looks the most like the concept car, and try and re-engineer it to fit this new concept. The likely outcome of this approach would be a dilution of the concept and a multitude of compromises.

Summary

I think that it is extremely important to work from a clear concept, and then deal with the specifics of implementation later. Taking what you have and changing it in a linear and incremental way in response to various different requirements will confuse the architecture of the system, involve complicated rework and result in a confused, complex solution.

The Dyson Design Philosophy

The Dyson Design Philosophy

Design isn't just about how something looks, but how it works. I don't see a difference between a designer and an engineer. A designer should be both. They should be building, testing and using the product, making it work better. James Dyson

This philosophy, when applied to vacuum cleaners, created a fresh product that was technically better, had a more contemporary, fashionable design and was also more user friendly. These product features were totally integrated, they depended on each other for the overall package. You couldn't design how the product should look, and then throw it over the fence to the engineers. It was an integrated, iterative process that incorporated all the aspects of design and engineering to create a successful product. The fashionable look came from the engineering possibilities discovered by the engineers as much as it was created by the designers. This philosophy is directly applicable to the problem of web application design and development.

The Four Box Model

If you don't mind over-simplification, you can divide most people involved in the creation of a web application into four basic areas, or boxes.

1. Creative


2. Web Designers


3. Internet Developers


4. Back End Service Developers

People rarely fit neatly into one of these boxes, but you can usually position somebody primarily in one box with an appreciation of adjoining boxes. For example, a web designer, as well as being an expert in XHTML, CSS, JavaScript, cross browser compatibility and accessibility, will probably have a good grounding in User Interface Design, and have some artistic ability and PhotoShop skills.

The Creative Silo

I have found that the first stages of a project are usually dominated by the Creative box with no input from Box 2. Ideas people in Marketing get together with artists and interaction designers. Sample customers are given pictures or Flash demos of various page options to make decisions on how a page should look. Interaction designs and specs are produced, and the artwork commissioned. Only after this lengthy process is completed do the Box 2 Web Designers get involved. Even though they might have really valuable input into the design and the way that the whole application might work, it is already too late in the process for them to have an influence.

Agile Case Study: Living the Dream...

Implementing Agile

Given my revelation about how Agile should be run without compromises and in a Lean and lightweight manner, I'm trying to run my current project in this way and seeing what gets in the way. I'm expecting the organisation, its culture or its people to put lots of blockers in the way, so I'm going to document how it goes in this article. But lets not start off with a negative attitude.

Kick-off

I'm kicking off the project with a Story-writing Workshop. I'm getting everyone who has a say in the project into one room and getting them to identify the customers of the system, and what their Role is. Then I'm going to get them to write all the stories they can think of from each Role's perspective. I'm not going to reject anything during the session, and I'm trying to keep the session quick and lightweight.

The Session

Got everyone to introduce themselves, and immediately I've got the advantage that everyone knows everyone involved in the project and their role. I got a load of pens and post-its and quickly introduced the concept of a Role as customers of the system. I asked everyone to write down the Roles they thought were relevant to this system.

I then introduced the concept of a Story as "As a [role] I want [a capability] to [deliver an outcome]. The Who, the What and the Why. I gave a couple of examples, then everyone got on with writing stories. After they had finished, I collected up all the post-its, said what the next stages were, and ended the meeting.

Processing the Information

During the session, I stuck all the Roles on a board and grouped them by role area. I included the duplicates to create hot spots that highlighted the most common roles. I noted them all down in Visio and binned the post-its.

I then took all the stories and put them in Word, and each time a Role was used, put a mark by that Role. This helped to identify the Roles that had the most relevance to the system. After consolidating the stories and organising them by Role, I colour coded the Roles Visio diagram with red for the most used Roles down through orange, yellow to blue for those roles that didn't have any stories put forward for them.

Clarifying Stories

I went through the stories, consolidated them and ordered them by role. This made it easier to identify the real roles that were important to the system, and I tweaked the roles in the stories so that they were consistent with each other. I put the roles at the top of my story document and described each role. We now have a fairly sensible set of stories, and we are fairly clear on the roles.

Release Plan

As part of the workshop, one interesting piece of information that I uncovered was that we need something in live in three weeks. This puts us under extreme pressure to deliver straight away, which is not really what is needed when you're trying to guide everyone through a process that is new to them.

We need to put high level estimates by each story to allow us to do the initial prioritisation and release planning. This is where things start to get tough. Various stories involve lots of existing products and processes that need to work together in new ways, or have the new system integrate with the current processes. To make a sensible estimate, even at a high level, we need to understand these other systems and processes. We need to talk to about three or four different technical areas within the organisation, but these people aren't available. We need to get our Truth, or Customer to clarify their stories, but they aren't available for a few days either. We need a Project Manager to pull things together, and a Dev Lead to arrange the technical meetings. None of them are available. We can't finalize the stories, we can't estimate or prioritise them and therefore we can't start work.

The Deadline

The business are extremely focused on getting something out by the end of the month, and will be annoyed if they can't have it because of the time-sensitive nature of the business case. We are losing days trying to get the information that we need to start the project. It is already July 7th. We will need a week to progress through the test environments and into live. Allowing a couple of days for system testing before we progress into the pre-live environments, this leaves us with 8 working days to develop the whole system.

Given the number of elements required just to fulfill the most basic system, we obviously can't hit this deadline. However, I find that promises were made to the business by various people in Development during the Propositions process, assuring them that something could be live by the end of the month. Various solutions had already been initiated prematurely without clear requirements or a release plan. I can feel that people are getting impatient, and the perception is that the process is delaying things.

The End

By the time I get in on Monday morning, the Business have demanded that Development keep their promise of hitting the end of the month deadline. Things go on over my head, and the conclusion is that the the project must continue using the JFDI methodology. I step aside, disappointed but not at all surprised. The company has rejected Agile and XP, and that's a real shame. However, its better to have this confirmed than labour under a misconception.

Agile - Seeing the Light

Agile with Compromises

Until last week, I thought that we'd implemented the Agile process pretty well given the organisational constraints we have. OK, we had to make the odd compromise from the Kent Beck style vision - but we were doing TDD, pair programming, acceptance tests, stories, iterations etc. We were delivering projects and it seemed like a success.

Seeing the Light

A week away as a group of 4 developers and a tester. A room with a handful of dev PC's that are shared. A fresh environment set up with everything configured in the same way. A whiteboard or two.

The customer says what they want. I write this as a story on the whiteboard. We agree on the tasks. We pair up and take the tasks on in order, putting our names by the tasks as we take them on. We switch pairs every few hours. We talk all the time. There are no interuptions. We get through a huge amount of work, and we all know the system at the end. We document the significant details online as we code. We enjoy it, there's free stuff to keep us entertained, we have specialists to ask when we need to get something resolved. We complete the story in a couple of days... That was easy! Why is it so tough to get anything done at the office?

Compromising Agile Principles Impacts Productivity

Working in a fairly standard, open-plan office environment with typical constraints imposed by the organisation appears to adversely affect our productivity. One factor I have identified for this is that other considerations have led us to compromise on many of our Agile practices.

• We have compromised quite heavily on a single customer always being available to the dev team. We have many customers for each project, and they are rarely available.


• We have lost some of our confidence to refactor our codebase, as the business perceives this as an unnecessary risk.


• Architecture, Security and Operations aren't involved on a day-to-day basis with the project team, so can't get their requriements fulfiled as part of the project. This means that they either do a Big Design Up Front, or play the part of Guardian of the Live Environments, stopping projects that don't fit their strategy. They are given a notional say in the project as 'customers', but aren't generally available to be part of the project team.


• In XP/Agile, no functionality should be added early. However, there are departments that have long-term plans that try and get their solutions implemented a step at a time, but without delivering anything tangible until towards the end of the development effort.


• Automated acceptance tests have been done, but can't keep up with new developments due to a lack of skills in the testing tool.


• Moving people around: The company has been structured so that there is little scope to move people around. Moving people around various teams and areas of the system helps to avoid knowledge loss and coding bottle necks. Even if people have the skills to cross the artificial boundaries, they are prevented from doing so by departmental constraints.


• Release planning is very complicated due to slots available for testing, business delivery deadlines, integration concerns. Therefore we usually have a drop-dead date to hit, and usually with a set deliverable. Not at all Agile.


• Solutions and timescales are coming through the Propositions process before any dev team has visibility of the project. This restricts the flexibility that the team has to deliver solutions in an Agile way.


• The tools we use to record stories and tests make it a significant effort to write stories, tasks and iterations, move things around, assign things to people and keep everything up-to-date. Things tend to get out of date due to this overhead.


• Effective communication: We are communicating using documentation far too much. Documents take a lot of effort to write and keep up-to-date and as a result are usually out-of-date. Far richer communication can take place person-to-person.

The Office Environment Impacts Productivity

Look at what you do every day in the office. How much of it is actually delivering value to the business, and how much is just noise? Its quite enlightening when you look at a breakdown of your day and see how much of your time and effort just gets dispersed into the company without anything to show for it. The Lean Software Development approach is to cut out as much of this waste as possible and concentrate your efforts on delivering value. There are always 'good' reasons to take on the tasks that don't add value, but it doesn't take many of these before they start putting real barriers in the way of doing your job. It doesn't take that long to fill in this form, to go to that meeting. Just a quick chat here, a quick chat there. A firewall preventing you accessing your dev environment, an urgent bug to fix, a problem to head off, a problem with a tool you are using, raising a call with IT support. Suddenly, 80% of your effort has been dispersed and you only have 20% left for the project. The Lean philosophy is to remove as much of this waste as possible.

Team Selection Impacts Productivity

You need to get the right team together to work well in an Agile way. They all need to be very good developers who will write quality code and communicate well. You need to know the skill set of the people that work in the team and ensure that you've got decent coverage of the areas that the project will need to develop for the project. There will be an overlap between the skills, but you should make sure that at least one person in the pair is skilled in the right areas for that particular story. If there is any specialist knowledge required that isn't held within the team, then bring that specialist in to work with the team for a while.

Agile Without Compromises

In our training lab we had the right team of people, the right office environment, the right development environment and a no-compromises Agile process. With all these elements in place, it just worked. It was easy, it seemed the obvious way of working, and nobody was constantly questioning the process. The goal was clear and well-defined by the customer. There were no barriers between the project team and their goal. We were motivated and our workrate increased accordingly. I realised that, in gradually allowing certain elements of the Agile process to be eroded by making compromises, the value of it was severely diminished. These processes and practices support each other and work together to allow the rapid development of simple, quality solutions that deliver business value, quickly. If you start meddling with any part of it, you will expose a weakness with another. So how do we do Agile/XP/Lean Software Development properly, and get the full value?

One Team

It is important that departments such as Security, Architecture and Operations are involved in the development of the solution. However, this involvement needs to be focussed and well managed. They should be part of the project team, with an interest in the project being a success, rather than taking on the role of customers or gatekeepers. They are probably best utilised as specialist consultants, who have all the knowledge of their area, and they bring this to the projects rather than publishing documents and constraining projects.

Good Communication

Agile is about people communicating directly in preference to through documentation. The reason for this is you get for better 'broadband' communication person-to-person over the 'narrow-band' communication through documents. Don't take this as 'Agile projects don't need documentation'. They do. Just document the important stuff, communicate the rest person-to-person.

No Interruptions

If a team can get together and concentrate on a project without interruptions, that will increase productivity massively. When somebody gets interrupted, its not only the time required to deal with the query that is the problem, its the constant context-switching and breaking concentration that has the biggest impact. Open-plan offices can be great, but the ideal for a project team is a room to themselves.

Good Development Environment

Having good, robust environments available, easily refreshed and with shared, high-spec dev PC's is essential. Developer access to all dev boxes is also important. Every time you hit a firewall or permissions problem, is saps time, energy and enthusiasm. You need access to your dev environments.

Summary

There are many parts of Agile that are just good programming practices that can be utilised in any process, for example Unit Testing, Continuous Integration and Automated User Acceptance Testing. However, to derive the full business value, quality, speed and flexibility of Agile and XP, you need everything to work together with the right people in the right environment, with a minimum number of constraints and barriers between them and their goal.

Glass Wall Anti-Pattern

Glass Walls

Nigel has come up with a concept of a Glass Wall, and it really struck a chord with me. You are expending a lot of energy to reach a goal. You can see your goal clearly, and you can see how to get there - its only a few steps away. After a disproportionate amount of time and effort, you find yourself in a totally unintended place, and with no real idea why. You haven't achieved your goal, but you've expended all the time and energy you have available for that task. When someone asks you why you haven't just taken the few simple steps to reach your intended destination, you find that you can't answer. Its those Glass Walls. But what puts those Glass Walls in the way? Is is the Malignant Requirement and the Headless Chicken ganging up on you?

Financial Model For Architecture

The Question

Should Architecture have a budget of its own? If you give Architecture all the money and power, they tend to spend it all building ivory towers and inventing wonderful architectures that only live on Visio diagrams, while the rest of Technology is busy delivering 'tactical' solutions on a shoe-string. This means that no architectural considerations are taken into account when building solutions. If projects have all the money, then they want the quickest, cheapest solution - they don't care about building the architecture. It isn't looking good for architecture.

Creating a balance

The only way to ensure that projects are delivered quickly and cheaply, but also to architectural standards is to create a balance of power. Architecture can take advantage of economies of scale. They care about the infrastructure, the non-functionals - I see these in the horizontal plane. Projects care about delivering capabilities, the functionals - I see these in the vertical plane. Their primary concerns are orthogonal - they usually appear to be in conflict, but they are actually supporting each other. What we need is a way for architecture to be able to invest in the infrastructure in a way that supports the projects.

The Financial Model

No project wants to take on the financial burden of being the first one to use a new, expensive piece of hardware or software that every other project is going to come along and use for free. Therefore they will avoid using the most architecturally sound solution on the grounds of cost and time-to-market. I propose a financial model, where architecture has a virtual budget. They can invest in whatever infrastructure they want, but at the end of the year they have to break even.

An Example

They initially get into debt by investing in a centralised customer database, because they know that there is a project in the propositions pipeline that will need this capability. It costs them £80,000, and they have it up and running as the project enters its implementation phase. They only charge the project £20,000 because they expect to have at least 5 other projects that will need to use it. In addition, they are making the project more efficient because they don't have to take on all the work to deliver a solution to store customer data.

Reasons Why

This solution is basically speculation in the infrastructure market. Architecture is the only department that can take a long term view. It needs to be able to invest in the infrastructure, because nobody else has responsibility for delivering it. However, they need to have a reason to invest in the right things at the right time to support projects, otherwise they will do what they want without consideration for project delivery. If the financial model forces them to work hand-in-hand with the projects, and encourages the projects to consider the best architectural solutions, the balance is there to ensure that effective, architecturally sound systems are build in support of project delivery.

Estimating - The Wicks Kitchen

Estimating

This ties in with the propositions process and estimating for projects. The idea is that you should be able to do a quick estimate to get a ballpark figure to work with so that the project has a good idea of the magnitude of the work. This can feed back into the priorisation and cost-benefit analysis, plus budgeting. However, this estimate should not be used for detailed planning, as its accuracy is only +/-50%, as there is limited information available to base the estimate on. You don't want to spend £50,000 on people's time to perform a detailed breakdown of how long a project will take. It won't be accurate, and you've already spend £50k that could have been spent on building the thing. Once everything is approved then you should enter a second phase of more detailed estimation (Iteration, Story, Task) until you have a task by task breakdown. This should be estimated in Story Points, and the true cost of the project will only become apparent as you monitor your Velocity and translate Story Points into mandays. This is hard to explain to a project manager, but if you provide them with the statistics and accurate predictions as the project team settles down, they will appreciate the benefits that this approach gives them.

The Wicks Kitchen

Nigel likens this process to the one he experienced at Wicks recently when ordering a kitchen.

Headless Chicken Mode

What is Headless Chicken Mode?

Headless Chicken Mode is a state of mind, and can sometimes seem like some sort of highly contagious mental condition that can spread through the office like wildfire. Anybody from architect to junior developer can suffer from this condition, and it is usually brought on by some pressure from the business. The further up the chain of command this condition permiates, the bigger the problem is. Symptoms include a fixed, slightly glazed stare, a sheen of sweat on the forehead, and rapid walking around the office talking to the wrong people about the wrong things at the wrong level, and lots of enthusiastic discussion about completely inappropriate solutions. The business and non-technical managers love this sort of thing. They see people jumping into action, reacting in a positive way to their motivational techniques, lots of activity and enthusiasm.

The Business Idea

The business has this fantiastic idea, and have convinced all the senior managers that this solution would really benefit the company. They build up huge enthusiasm for the idea, get loads of budget, print special T-shirts, have promotional events, put adverts on the big screens. And to be fair, their business outcome they are aiming for makes absolute sense. Then two days before their go-live date, they remember that somebody has to actually enable the systems to support this idea. But how hard can it be?

Top Priority - Process Ignored

They don't need to go through the normal 'propositions' process, so carefully put together by Technology, because the CEO has decreed that this is the priority and will be delivered. Therefore they approach a team leader, and make sure that he is fully aware of the absolute urgency of their request. They usually present a technical solution, rather than a business outcome, and give the absolute deadline of, say, two days. The team leader really doesn't want to be seen to fail given the high profile of the project, and to even appear unco-operative wouldn't be a good career move, so he immediately makes two serious errors. He implicitly accepts both the deadline and the technical solution put forward by the business.

Maintaining Control

You have various controls at your disposal when you are running a project. You can control the scope of the project, so you can choose to deliver more or less features. You can control the quality of the technical solution, but this is a very dangerous control to use. You can control the amount of resource allocated to the project, but you have to be aware that they gain from this isn't linear, and can actually cost you efficiency if you try to ramp up too quickly. You can control the time allocated to deliver the project. Finally you can choose to fulfil the business outcomes with a non-technical solution if that makes sense - and this will usually be invaluable if you have a tight deadline, but over the long term will usually not be the most appropriate solution.

Losing Control

The team leader has just given away scope, time and manual alternatives to the business. He hasn't got time to introduce a big team effectively, so he can't really use resource as a control either. As for quality, he hasn't discussed SLA's (Service Level Agreements), so they default to production quality, 99% uptime. So he now has no control left at all.

Switching to Headless Chicken Mode

This is when Headless Chicken Mode strikes, and he runs round the office in a mild panic, asking random developers if he can use this database to get this bit of information, if they could go direct to the database without having to build a web service, if they could build a web page in a couple of hours this afternoon. He rarely gives context, so nobody has the opportunity to suggest an alternative, or highlight the problems with this approch. Then he tries to find a fast track to live, skipping testing.

The Mythical Next Release

Many people who should know better do this sort of thing, and they always believe that they have no choice, and that its ok to deliver this complete rubbish, because 'the next release' will sort out all of its shortcomings. However, the next release never happens, because the business never spend money on making the technical solution better, they will be investing in their next idea. If the app keeps falling over, it is up to Operations to fix it. If its got bugs in it, then development will provide bug fixes. They see no point in spending any more money when they already have their application in live, no matter that it is costing Operations, Dev and Testing a huge amount to operate, test and maintain. It's not their fault that we can't do our jobs properly. But Technology has now got high operating costs, and their ability to respond to new ideas is compromised.

Avoiding Headless Chicken Moments

So how should we deal with this situation? Well, the key thing to remember is that it should always be a negotiation between technology and the business, neither should be able to dictate things to the other. In addition, the business should never present a technical solution; they should present the business outcomes that they want to achieve, and negotiate with Technology to agree on the most appropriate way fulfil these.

Iterative Delivery

If the deadline is unrealistic, first try and work out what actually needs delivering for that deadline. Usually you only need to deliver a small subset of the solution for that date, and then you can iteratively add features on a 'just-in-time' basis. The next step is to look at what manual solutions you could put into place to deliver the same business outcome while you deliver the automated solution. Could you set up a call centre with people manually entering data? Could you send emails to customers, send a form through the post? Place an advert in a magazine? You don't always need to deliver the technical solution.

Is It Worth It?

Another good discussion point is cost benefit. If you can demonstrate that it will cost X to deliver the automated system, and the predicted benefit isn't several times higher than this, is it worth doing at all? When discussing things with business people, always bring it down to money - then you're talking the same language.

Conclusion

Once you are aware of the Headless Chicken Mode, it is easy to spot. The look of mild panic is the first sign. If you then overhear snippets of conversation, or see an email trail that indicates that ill-conceived solutions are being hacked together for an unrealistic deadline, you've got a 'HCM' problem.You have to put a stop to it, or your technical platform will be corroded to the point that you will spend your time fire-fighting rather than producing inductry-leading solutions that you are proud of.

The 80/20 Rule

The Pareto Principle

The 80/20 rule, or the Pareto Principle asserts that 20% of something is always responsible for 80% of the results. This is a rule that can be applied to almost anything, and if you apply it appropriately, it can save you from investing time and effort in areas that give you diminishing returns. I mention it here because you can apply the 80/20 rule very effectively to software development, and I use it every day to guide my decisions on where to focus my effort and how to approach project work.

Requirements

So why don't we always just invest 20% of the effort to get 80% of the benefit? In my experience, if you ask people what they want from a given system, they will always ask for everything. They will spend weeks or months coming up with the most comprehensive list of requirements you could imagine. They will, as usual, try to tell you how to implement the system too.

Prioritisation

You can deal with the sheer mass of requirements by converting them first to high level outcomes. These outcomes can be prioritised. Then comes a period of negotiation, where you try and agree on the most appropriate way to deliver each outcome. Once this is done, you can create high level stories that you can estimate. Telling someone that feature X will cost £100,000 soon makes them focus on what is really important. You can then use this additional information to further prioritise these high level stories and see what you can produce for a sensible amount of money, in an iterative manner, achieving real benefits within an acceptable timeframe.

The Malignant Requirement

So we're doing pretty well - we have got a prioritised list of the best value stories that will deliver the outcomes that the business want. We've made sure that the solutions are appropriate, and we understand the impact on the existing infrastructure. Now let's implement that 20% to gain the 80% benefit and move on. But we can't. Why not? Because there is always one Malignant Requirement that the customer will not let go of. This will usually be some really strange, specific requirement, based on some historical pain that they have suffered as a result of an old system or process. Or it could be the sort of requirement that seems really important if you've spent weeks over-analysing the problem domain in isolation.

An Example

They might have had a lot of problems with timeouts in the past because the old system had really long, synchronous call chains. They worked out that it cost the company £500,000 over the past few years, so they are quite happy to spend a huge amount of money on a requirement to dynamically set all timeouts across all technologies and consume real-time data; monitoring each call with respect to the timeout it has been given. The new system has short call chains and an asynchronous architecture, but this isn't taken into account. The fact that dynamic timeouts and real-time monitoring and alerting will make the system far more complex and more expensive to maintain isn't seen either. This one requirement will mean that you put in 80% of the effort, and see 20% of the benefit.

How to Supress the Malignant Requirement

This is a tough one. Trying to educate the customer in the technical considerations is one approach, but this won't work unless they are technical in the first place. Risk and cost-benefit analysis is probably the way to go, and it really helps if you can get an idea of the impact that it will have on future development and maintainance costs. If it will cost £250,000 more over the next 5 years, then this will offset the benefit of the requirement. However you do it, it is really worthwhile trying to supress this requirement, or you lose your ability to capitalise on the 80/20 rule.

Achieving 80/20

Most problems have been solved many times before. You're not the first person to want to monitor the behaviour of your live machines. You're not the first person to want to take orders over the web. These problems have been solved over and over. People write frameworks for this sort of thing, so if you can't find what you need in the opensource community or from a vendor, you need to question your requirements. If your problem domain is really specialised, and the solution that you are looking at really is the first time the industry has encountered this problem, then go ahead and spend your time and resources on solving it. However, if you are writing your own framework for logging, or for website navigation, then question what you're doing. Research on the web. Google, newsgroups, technology sites, find out what's out there. Top technical minds will have spent years solving the problem that you've encountered. They will have learned from previous releases and incorporated this learning into the current release. Bugs will have been fixed, performance optimised. Do you really want to spend years reproducing all this work?

Don't Bespoke the Solution

Another critical point to understand is that if the solution solves 80% of your problems, and its only taken 20% of the effort to implement it, don't go and bespoke the system to try and achieve the remaining 20%. It might seem to make sense to begin with, but it will really hurt you in the longer term. By using a product or solution developed and maintained by a third party, you should be able to benefit from upgrades and bugfixes really cheaply. You might also be able to use third party tools and utilities that integrate with this product to give you instant benefits. If you go for this final 20% to fulfil the Malignant Requirement, you will lose a significant proportion of the 80% benefit you had in the first place. I see it time and time again, where a company has totally bespoked a third party product and then has to maintain and develop it itself, and are unable to upgrade and subsequently fall out of support. The development and maintenance costs are huge, and the system has to be eventually ditched and re-written, incurring further costs.

Conclusion

You can apply the 80/20 rule to all the areas of technology where the problem has been encountered before, enabling you to reuse the solutions that are already out there. The benefit of the snowballs when you look at the costs future development, maintenance and system integration. You should seriously question any requirements that aren't satisfied by a mature, industry-accepted solution. This frees up resource to focus on those areas that are specific to your business, your particular problem domain, and pushing your technology forwards to better serve the business.

Web Technologies

Introduction

When you want to provide services to customers over the internet, you are faced with a confusing array of options. The landscape is changing rapidly, and you have to make the call on which technology is the most appropriate for you. What are the options? What are the pros and cons of each? How do you decide which one you want? Here's the Dummies Guide to internet technology.

Standard HTML Web Pages

Starting with the humble HTML web page, you would think that you were on safe ground. Then people start mentioning W3C standards, XHTML, accessibility, cross browser compatibility and stylesheets (CSS). Anything other than the simplest page demands that you use javascript. So what are these things, and why do you need them?

W3C standards

The World Wide Web Consortium (W3C) is an organisation that creates standards and guidelines for the web, and most browser vendors produce 'standards' browsers that follow these (Firefox, Opera, Safari). Microsoft's Internet Explorer is the notable exception, unfortunately, but IE is improving its standards support with each release. It makes absolute sense to follow these standards rather than support a particular vendor's browser to the detriment of all others. Standards support is getting better, so the browsers will support your web pages better as time progresses. When there is a browser bug, it isn't unusual for people to demand that you, as author of the web page, fix it. It is a misunderstanding of how the web works, and especially how it will work over the next few years. You don't support a browser, the browser supports standards, which you adhere to. If the browser has a bug, and there isn't an elegant way to work around the problem, then it is up to the browser vendor to fix it. "Don't we support IE any more?" is a question that highlights a fundamental misunderstanding of the internet.

XHTML

So you know web pages are written in HTML. Well, now we have XHTML, and the benefit of this over HTML is that its far better defined, and can be described by an XML schema (hence the X prefix). The upshot of this is that browsers find it far easier to interpret, and therefore next generation browsers should have far better, more consistent support for XHTML web pages. As a result, cross-browser compitbility should improve, lessening development and maintenance overheads that we currently incur ensuring that our web pages work on all browsers. It takes less processing to read XHTML, so it works far better on browsers that run on mobile phones or PDA's. Basically, if your site is XHTML, you are expanding your user base and future-proofing your website.

Cascading Style Sheets (CSS)

Cascading Style Sheets (CSS) allow you to seperate your web page's document structure and data from its presentation data. You create a central style sheet that gives you the ability to make global changes to the look and feel of your website. If you want all buttons to be red instead of blue, its on quick change to the CSS file. If you want to dynamically rebrand your site depending who is viewing it, you can do that too. Although it isn't the work of a moment to build CSS that works across all browsers, once it is in place, writing brand-complient web pages becomes a doddle, as you just have to write the correct document structure, and the page will automatically conform to brand guidelines. Investing in CSS, seperating out the presentation information, will save you when it comes to development and maintenance and rebranding.

Accessibility

Accessibility of web pages means writing them in such a way as to allow everyone to access them. People who have difficulty reading a monitor should be able to use a screen-reader to read your web pages and give a good user experience. Turning images off should not compromise the user experience. Using a different browser or device should not make the site inaccessible. And more controversially, turning off javascript should not deny the user access to the site.
Following web standards makes your site accessible, see the guidelines on the W3C site.
The basic motivation for making your website accessible is to make it available to the widest possible audience, but more crucially, not to discriminate against anyone. If you do discriminate against somebody by not making your website accessible, you leave your company open to legal action. Motivation enough, I think you'll agree...

Flash, Flex and Rich Internet Applications (RIA)

Adobe's (formerly Macromedia) Flash player allows you to provide a richer graphical experience using the FlashPlayer browser plugin, and also allows you to stream music and video. The term RIA (Rich Internet Application) refers to a browser based application that takes on more of the UI processing at the client side, therefore running something approaching a desktop application within the browser. However, most of the data and state information is still held at server side and is accessed via the internet. Adobe's Flex offering provides a toolkit to enable development of RIAs.

Smart Clients

If we take this transfer of responsibility to its natural conclusion, Smart Clients are applications that are installed on the user's desktop with access to local resources (memory, disk, database etc), and merely accesses data from the internet when necessary. This is where the industry is heading in the next few years.

Web Services

If you want to access data in a standards compliant way across the internet from your website, RIA or Smart Client, then you will be needing Web Services. Web Services are a standards complient way to offer services securely over the internet. The W3C have produced the WS-* standards, and various implementations of these include Microsoft's WSE2, WSE3 and Indigo and Apache's Java based Axis.

Model View Controller

What is MVC?

When developing a web application that has anything other than the simplest page navigation, it is extremely important to seperate the presentation, the business logic and the control of the page-flow. The best way to do this is by using the Model View Controller pattern (MVC).
The 'Model' is the part of the system that deals with the business logic, it is the logical representation of the problem domain. In OO this would give your abstrations such as Employee, Customer, Payslip etc.
The 'View' refers to the web page layout and presentation of the information to the user. In a web application, this is usually represented by markup.
The 'Controller' is the part of the web application that takes an event (usually a user action) and decides how the system should respond to it. It performs any updates that are required to the Model, and then decides which page (View) to display next.

MVC in J2ee

This pattern is implemented in the Apache Foundation's Struts project, an opensource Java framework. The specific pattern is called MVC II, which is a Front Controller implementation. A Front Controller is where the Controller, not the View, is the first to receive the user action and then decides what actions to take in response. Once it has performed any actions it needs to in response to the event, it decides which page to display. However, a particular event isn't tightly coupled with the page is displays, and to ensure that you can easily change the destination page, each page is responsible for getting any information it needs to display from the Model, and this information is not passed by the Controller to the View.

MVC in ASP.NET

In my opinion, the MVC II model is the best option for any enterprise scale web application. However, I tried to apply this to an ASP.NET project, and I found myself battling against language features every step of the way. The main issue is that ASP.NET by default employs the Page Controller model, where Postbacks are used to enable the page to deal with any events that result from user interaction with any of its controls. Therefore you often end up in the awkward position of handling the Postback in page X , and in response to that action, you actually want to be displaying page Y. It appears that, unless you want to take on the additional complexity of Microsoft's User Interface Process Application Block, it makes sense to accept that a lot of the language features demand that you use the Page Controller pattern, and try to make the best of it. I have read several Microsoft articles that claim that the ASP.NET page code-behind feature seperates the View (the markup) from the Controller logic (code-behind). This is not really true, as the code-behind contains a muddle of View logic and Controller logic. To ensure that you get an MVC seperation of concerns, the code-behind should contain only 'View' logic, and the Controller logic needs to be seperated out from this. As the ASP.NET model means that the events are handled in the code-behind, you end up with the View handling events. This is not really in keeping with the MVC model we wanted to implement.

Model View Presenter

As a result of these ASP.NET language features, I discovered that Model View Presenter (MVP) was the best pattern to employ when using ASP.NET, as it separates the behavior of a presentation from the view whilst allowing the View to receive user events. See Martin Fowler's definition, and a good example implementation by Jeremy D. Miller. Basically, the MVP pattern has the ASP.NET page (including code behind) as the View, its has the Model behind a Facade, and the Presenter object acts as the go-between that gets events from the view and performs the necessary interactions with the Model, and then informs the View with the results. MVP also aids the unit testing of the page logic, which is very cumbersome to attempt using NUnit when you are relying on ASP.NET runtime infrastructure to hook the page together.

Mocking Basics

Introduction

As an XP team that uses NUnit and Continuous Integration, we had the problem that our unit tests ended up being integration tests, and any error would propagate causing many tests to fail and making it hard to find where the original error occurred.
To solve this problem we used NMock to mock out any external dependencies that a class has, therefore making our unit tests insular. However, there are several patterns that we have to use to make this work effectively, and this is the topic addressed in this document.

NMock

NMock is a dynamic mock-object library for .NET. Its purpose is to provide a clean API for rapidly creating mock implementations of custom objects and verifying that objects interact with them correctly from unit tests.

What is a mock?

A mock:
takes on the interface of another object, allowing it to be substituted for a real one for testing purposes.
allows Expectations to be set up, specifying how the class under test is expected to interact with the mock.
fails the test if any of the expectations are violated.
can also act as a stub, allowing the test to specify objects to be returned from mocked methods.

Defining a Dynamic Mock

A dynamic mock sets up a dynamic proxy that implements a given interface, or overrides any virtual methods on a given object type. It is initialised as follows:
IMock mockObject = new DynamicMock( typeof( Interface ));
This mock object can take the place of the real object in unit tests and allow you to control the way that the mock behaves and verify the way the object you are testing interacts with it.

Setting Up a Mock

In order to verify the way that the object under test interacts with our mock we can set up expectations on our mock object.
mockObject.Expect("MethodOnMock", param1, param2, ...);
Here we tell the mock to expect a call to method MethodOnMock with the given parameters. An extension to this is the ability to return an object when a method is called.
mockObject.ExpectAndReturn("MethodOnMock", returnObject, param1, param2, ...);
Here we instruct our mock to return returnObject when MethodOnMock is called. There are several other expectations that you can set up, which we cover in the Appendix.

Verifying a Mock

Once you have run the scenario that should fulfil the expectations that you have set up on the mock object, you have to verify that this is the case.
mockObject.Verify();
If the expectations have not been fulfilled this will fail the test.

Passing a mock to a method

Pattern 1

Whenever the class that you are testing instantiates an external class, always acquire it through a virtual property or method; eg

public void Method() {
Interface object = ObjectInstance;
}

protected virtual Interface ObjectInstance {
get{ return new Object(); }
}

This means that whenever you interact with an external class, in the test you can override this property or method and return a mock object in its place. The simplest way of doing this is passing your mock into the constructor of your test extension and storing it for your overridden property or method to return. The unit test code will look something like this:

[TestFixture]
public class TestClass {

private ClassTestExtension _testObject;
private IMock _mockObject;

[SetUp]
public void SetUp() {
_mockObject = new DynamicMock( Interface );
_testObject = new ClassTestExtension((Interface)mockObject.MockInstance);
}

[Test]
public void SampleTest() {
_mockObject.Expect("Method");
_testObject.MethodUnderTest();
_mockObject.Verify();
}

private class ClassTestExtension {

Interface _mockObject;

public ClassTestExtension( Interface mockObject ) {
_mockObject = mockObject;
}

protected override Interface ObjectInstance {
get{ return _mockObject; }
}
}
}


Rules of thumb

If a class depends on any other class to perform any of its tasks then do the following:
In the NUnit test, derive from the class you are testing to give you control over the external classes it uses.
In the base implementation of the class you are testing, every time you need to use an external class, get it via a virtual property.
If you own the external classes, make it implement an interface to aid with mocking.
In the inherited test class, override these properties to return mock instances of these classes.
In the derived class, overload the constructor to provide one that takes the required mock objects and stores them as member variables so that they can be returned by the properties.
In the setup of the test, create a new instance of the derived class and all the mock objects, passing the mock objects into the constructor. Store all of these objects as member variables.
In each test set up the mock objects (ExpectAndReturn etc) to provide the scenario that you want.
Call Verify on your mock objects at the end of each test.