Track Project through Microblogging

During a project, lots of meeting occur, decisions are made, actions are given out and completed, accountabilities are assigned. Its hard to keep track of all of this over a period of months, and usually ends up with people searching their email and filesystem, and asking people who were in the same meeting or workshop to find evidence of something that happened, or a decision that was made.

My idea is to have an internal twitter-like service that you enter key things that happen on a project so you query them later.

http://status.net/ is a twitter-like service that you can install internally to your organisation or have a private instance in the cloud.

Nanoformats define here need to be extended to focus on things that happen on a project eg action statuses etc:

http://microformats.org/wiki/microblogging-nanoformats

* tag++ (discussion)
o Used to 'plusplus' (vote for) a user or subject
o Example: microformats++ because they rock the disco!
o Syntax: ++
o Tools: plusplus bot
* tag-- (discussion)
o Used to 'minusminus' (vote against) a user or subject
o Example: drm-- for breaking my backups
o Syntax: --
o Tools: plusplus bot

1nanoformats (syntaxparam)
* @username
o Used to mention a user.
o Example: @mike I agree with you
o Syntax: @
o Tools: @yourusername filter pipe, foamee
* #tag
o Used to tag posts
o Example: kauta concert at Doka #music
o Example: #rockclub kauta concert tonight (group use example)
o Syntax: #
o Tools: #tag filter pipe
2nanoformats (syntax:param)
* lang:
o Used to define the language of the post.
o Example: this is a post in English lang:en
o Syntax: lang:
+ info: code list iso639-1
o Tools: lang filter pipe
* L:
o Used to define a location
o May be followed by a trailing colon to indicate the end of the location (Example 4)
o Example: Good place to visit L: Place Georges Pompidou, Paris, France
o Example2: Good place to visit L: Place Georges Pompidou, Paris, 75004
o Example3: Good place to visit L:48.860500,2.352260
o Example4: L:San Francisco: is a nice place.
o Text Syntax: L: or L: at the end of a tweet
+ info Twittervision API, Google maps
o Geo Syntax: L:
+ info: (Section 3.4.2 of RFC2426)
o Tools: Twittervision, Twittervision API, twibble, a location aware twitter client and API, location filter pipe
* todo:
o Used to define something to do (or for someone combined with @username).
o Example: todo:go to buy chocolate
o Syntax: todo:
o Tools: todo filter pipe
* note:
o Used to write a simple note.
o Example: note:printer ink catridge numbers HP336 / HP342
o Syntax: note:
* event:
o Used to define an event (better combined with L:).
o Example: event:Conference or event:http://www.web2summit.com/pub/w/40/coverage.html/
o Syntax: event: or event:
o Tools: event filter pipe
* enclosure: [play:(deprecated)]
o Used to mark a link to playable media.
o Example: enclosure: http://media.mfdz.com/track/7.mp3
o Syntax: enclosure:
* time:
o Used to define a time (usually with event).
o Example: time:2006-06-30 21:00:00
o Syntax: to define (can change)
3nanoformats (syntax:param:param)
* rate:
o Used for rate something between 1-5.
o Example: rate:1:Epic movie or rate:5:http://www.twitter.com/
o Syntax: rate:<1-5>:
o Tools: rate filter pipe
* vote:
o Used for vote (for, against, abstain).
o Example: vote:against:Irak war or vote:for:Peace
o Syntax: vote::
o Tools: vote filter pipe
* deal:
o Used for transaction info (sell, rent, trade, wanted, offer, announce).
o Example: deal:sell:acer:n50 pda or deal:wanted:http://www.nabaztag.com
o Syntax: deal::

Web App Development Methodology

System Design and Web Design

System Design

There are many methodologies that deal well with clarifying requirements and then designing and building systems to satisfy those requirements. You have your use cases, stories, functional specs. You create logical designs that model the domain, examine how actors interact with the various systems to fulfil a business requirement. You can then start building and testing your code. Somewhere along the way, some web pages or windows forms get created out the back of this process. If developers have their way, they are usually logically correct, but terrible to use. But they do the job - don't they?
What do you mean there's a high drop-out rate?
How did I devalue the brand again?

Web Design

The other perspective that web app development gets approached from is UI design. The business go to the designers with some high level requirements, and they start the page design. Usability tests, rich interactions, brand, UI design principles, Flash demos, sitemaps, user flows, copywriting, graphic design, detailed UI specs; these all feed into a wonderful, engaging, on-brand customer experience. And then you build that code stuff on the back of the screens. I mean, the screens are the most important part of the whole thing.
Aren't they?
What do you mean you need some requirements?
And now you need to change the page flow...?

The Train Wreck

Usually these two flows happen in parallel, on the whole independently, and then towards the end of the project, there's a big crash when the two worlds collide.

What we need is...

What we need is a methodology that allows these two, largely disparate disciplines to work in harmony. User requirements need to feed into use cases which need to drive both the customer interactions with the system (usually web pages) as well as the interaction between various systems and business processes. The problem is, I'm having real trouble finding anywhere where this is taken into account. It generally seems to be either the web designer's approach or the system designer's approach. However, to be effective at writing enterprise-scale web applications with high quality systems and also a rich and engaging customer experience this is an absolute must.

High Level Idea

So how might this work? Taking the Unified Process as a starting point, we really should start out by clarifying the requirements. This is before any design starts from any perspective. Then out of the back of this you will get the use case diagrams which show the interactions between actors and systems. Customers can interact with systems via many channels: phone, web, letter, fax. These all need to be clarified and mapped onto the business processes. This should give you an idea of what the interactions are, what channel they use, and what data is required both in and out. Then you can start the system design and the web page design. There will be parts of the use cases where the actor is the customer and the channel is the web. These will give you the interactions and data flow required to start designing the customer interactions required on the website. You can then produce wireframes from these and subsequently add page-level detail, rich interactions and perform usability studies. All based on the use cases. It seems quite simple - does anyone actually do this?

Links

• Successful Development


• Successful Development 2


• Software Development Methodology


• Web Project


• Amazon


• Amazon


• Web Design Process


• Project Management


• Web Project Template Files


• System Design

More about the Coffee Cart

Delivering Coffee

Nigel Hamer modelled how IT delivers solutions to the business by looking how staff at a coffee cart dealt with customers. What interests me is how its delivered. The process.

Synchronous and Asynchronous Processing

The process of getting a coffee is something like this:

1. take order


2. take payment


3. make espresso


4. pour milk and espresso


5. hand over coffee

As I queue at the coffee cart fairly frequently, I started to notice that they had various ways of working. The two key ways were:

1. Asynchronous: Each person has a specific task (eg till, making espresso, heating milk, pouring coffee) and they communicated the order (asynchronously) using till receipts and writing "D" on the cup for decaf. The receipt and cups are Kanban in the Toyota Production System.


2. Synchronous: Each member of staff is responsible for a given order; they take order, then the payment , then make the coffee and then hand it to their customer.

There are various pros and cons of these methods. In the first, you are relying on the process and the communication mechanism, or "Kanban", to ensure that an order is fulfilled. If something goes AWOL, then the only way that they get notified of the problem is via the customer, and the person pouring the coffee won't have knowledge of what should have happened. However, as the process rarely goes wrong, sorting these problems out only take a very small overhead. At the coffee cart, if something goes wrong there is usually a degree of confusion, so you should have a well defined exceptions procedure, and a way to communicate the problem back up the process.

Using the second approach, you get somebody taking care of your order, so the scope for something to go wrong is reduced, and the resolution path is easier, as the customer refuses to take the coffee, and the member of staff dealing with that order can resolve the problem. However, the individual tasks will be less efficient, as there's nobody ensuring that there's always hot milk and espresso available, and the lead times of each may impact the delivery time of a particular order.

Choosing the right process is down to your requirements. If the asynchronous approach gives you a satisfactory success rate, even though there is no guarantee that each person gets the right coffee, if it works 99.9% of the time, it is far more efficient and gives quicker service. If you want to be very sure that each customer gets better individual attention, and a higher probability that they get the right order, and don't mind slower service as a result, then you choose the second.

The decision-making process normally seems to work the other way in the real world. People assume that 99.9% success, even with an exceptions process, isn't good enough, and happily take the hit on efficiency to get this percieved guarantee of successful delivery.

I would suggest that the first process, when applied to computer systems, will usually provide the best solution, with a perfectly good success rate.

Preventing Multiple Form Submits

Synchronizer Token Pattern

The Synchronizer Token Pattern is a server-side way to prevent multiple form submissions which is widely used in J2ee.

• Synchronizer Token Pattern (java world)


• Synchronizer Token Pattern (Core J2EE Patterns)


• Synchronizer Token Pattern (javajunkies)

Extract from Core J2EE Patterns

Duplicate Form Submissions

Users working in a browser client environment may use the Back button and inadvertently resubmit the same form they had previously submitted, possibly invoking a duplicate transaction. Similarly, a user might click the Stop button on the browser before receiving a confirmation page, and subsequently resubmit the same form. In most cases, we want to trap and disallow these duplicate submissions, and using a controlling servlet provides a control point for addressing this problem.

Synchronizer (or Déjà vu) Token

This strategy addresses the problem of duplicate form submissions. A synchronizer token is set in a user's session and included with each form returned to the client. When that form is submitted, the synchronizer token in the form is compared to the synchronizer token in the session. The tokens should match the first time the form is submitted. If the tokens do not match, then the form submission may be disallowed and an error returned to the user. Token mismatch may occur when the user submits a form, then clicks the Back button in the browser and attempts to resubmit the same form.

On the other hand, if the two token values match, then we are confident that the flow of control is exactly as expected. At this point, the token value in the session is modified to a new value and the form submission is accepted.

You may also use this strategy to control direct browser access to certain pages, as described in the sections on resource guards. For example, assume a user bookmarks page A of an application, where page A should only be accessed from page B and C. When the user selects page A via the bookmark, the page is accessed out of order and the synchronizer token will be in an unsynchronized state, or it may not exist at all. Either way, the access can be disallowed if desired.

ASP.NET client-side solution

The common way used to prevent multiple form submissions in ASP.NET seems to be to use javascript to disable the button on the page. This is not an ideal solution, with many drawbacks, but will work in the majority of situations.

Disable button (blog)

The Honda Way

Introduction

When there are many people in an organisation that don't want to work in an Agile way, they would obviously like to see the approach fail so that their way of working will prevail. This puts a lot of pressure on the Agile process to work perfectly, and for there to be no errors. This means that a risk-averse attitude is adopted, which goes against the concepts of 'refactor mercilessly' and 'have courage'. It also means that any worthwhile new ideas or techniques will be put to one side if there is any perceived risk involved in trying to progress.

The Honda ethos disapproves strongly of this, as recently re-iterated by their F1 Racing senior technical director Shuhei Nakamoto.

He encourages the team not to be afraid of making mistakes - because that is the only way the team will progress. He believes that a philosophy of experiment encourages the team's younger staff members to come up with fresh ideas - rather than being afraid to put forward concepts for fear of failure.

We flattened the organisation so that everyone working in (each) department can talk freely about their ideas and opinions. As long as an idea is theoretically correct and considered useful, we will discuss it - even if it comes from an inexperienced staff member. But they have to take responsibility for the ideas they provide. That is the Honda way.

Another thing I always tell the young staff is that they shouldn't be afraid of making mistakes. The failures will eventually improve our know-how. The important thing is to use that know-how correctly to make the next step forward. I kept telling them this. As a result, young people started to speak up and contribute more, which I believe was reflected in the gradual improvement in our performance.

I am happy about this, because it provides proof that the Honda Spirit of 'never be afraid, never give up' is penetrated thoroughly into the team. What Honda disapproves of is 'no action, no error'. I want all our staff to always keep the philosophy in mind.

How to Write a User Story

So What is a User Story?

A User Story is used in eXtreme Programming (XP) to define what the customer wants the system to do for them. They replace the traditional requirement specs and use case diagrams.

The Who, the What and the Why

There are many ways to approach writing a User Story. The following approach has worked best for me:-

As [the customer] I want [a capability] so that I can [achieve an outcome]

Its the WHO, the WHAT and the WHY. Who wants this capability, what is it, and why do they want it? It is useful to define the Who and the Why because this makes you consider who wants this feature and what you will achieve with it. If you have difficulty defining either of these, then you probably don't need the feature in the first place.

The Five Whys

To get to the actual, core reason that you require the story, it is sometimes necessary to peel back a few layers. A technique called 'The Five Whys', commonly used to fine root causes of problems, can be useful here. A similar approach can also be applied to finding the actual end-consumer for a story.

Capture the Essence of the Requirement

The Story does not have to, and should not, fully explain every last detail of the requirements. It should be brief, and form a hub for discussion and collaboration. A Story also has acceptance tests and tasks associated with it. The acceptance tests are what fully define the story. To enable you to frame the problem and define the Story to a level where it can be estimated and prioritied, you will probably need a few, high level acceptance tests. You can then get more specific and fine-grained in your tests during development.

Ditch the Jargon

Most Stories should be non-technical, they are capturing a requirement, not the technical implementation of the solution. The alarm bells should go off if there is technical jargon in the Story text. Avoiding describing the solution in the Story also gives you more options, such as solving the problem with a manual process, or a different automated solution that becomes apparent during discussions with developers and testers.

Vertical Slices through the System

It is important that each story represents a slice through the system, eg web page, web service, database. Splitting Stories horizontally, by technical area or layer in the system should be avoided.

Example

As the stock delivery manager, I want to be informed when any stock falls below its threshold level so that I can order more stock in before the store runs out.

Acceptance Tests and Tasks would be added during the Getting the Story Straight and Planning Game sessions.

INVEST in Good Stories, and SMART Tasks
Writing Good User Stories
User Stories

Project Roles

Roles Within Project Teams

For a team to work effectively, they need clearly defined roles, and individuals need to be empowered to fulfil these roles.

Project Manager (PM)

• Budget holder


• Owns overall project plan with timescales, deliverables & outcomes.


• Understands high level project progress and issues


• Provides environment to enable team to deliver


• Shields team from details of the business and politics.


• Deals with resourcing requirements


• Represents the project to senior management


• Works with Development Managers and Dev Leads to pull together the required proposition and take it into delivery if appropriate.

Empowerment Required: Demand that their outcomes are achieved by the dev team.

Team Leader

• People manager with technical background


• Runs sessions (eg standup, retrospective, planning)


• Supports and enforces development process by dealing with
  
  
  
  • people management
  
  
  • planning
  
  
  • progress measurment
  
  
  • external relationships
  
  
  • company politics
  
  
  • communication channels
  
  


• Works closely with dev team - especially Team Leader


• Attends relevant external meetings on behalf of team


• Ensures that the team operates smoothly, and have the tools & resource to deliver the project outcomes.


• Deals with issues that need resolving within and outside the team.


• Understands the technical details more than PM, but not to the depth of the senior developers.

Empowerment Required:

• Represents team


• Authority figure within team


• Runs project day-to-day


• PM primarily interacts with team via Team Leader.

Truth

• Provides the single point of requirements for the development team.


• Is able to answer all questions relating to requirements in a timely manner.


• Is available to the team at all times.

Empowerment Required:

• Authority to answer questions on requirements.


• Authority to make judgement calls on-the-spot.


• Empowered to commit all their time to the project and team.


• One Source for project requirements.

Lead Tester (QA)

• Works closely with Truth and Developers to understand requirements and provide acceptance tests, both manual and automated.


• Runs tests and sign off each story, iteration and release.


• Raises bugs.

Empowerment Required

• The acceptance tests they create are the single source of the requirements.


• If Truth is unavailable within acceptable timeframe, has authority to make the decision.


• Power to sign-off or reject stories, iteration and releases.

Marketing/Business

• Combines business knowledge and market research to generate initial high-level ideas, outcomes and goals.


• Sells the idea to the business and gets budget allocated to programme.


• Understands the long-term plan


• Generates the outcomes that are passed to the project team to fulfil.


• Understands how the outcomes fulfil the business requirements

Empowerment Required: To demand that their outcomes are achieved.

Technical Lead (Coach)

• The most senior developer on the team


• Takes responsibility for technical decisions on the project in conjunction with Architecture and other departments.


• Responsible for the technical output of the team, and ensuring that it complies with both internal and industry standards and best practices.


• Responsible for ensuring that the technical practices that support the process are implemented within the team eg Test Driven Development, Pairing, Continuous Integration.


• Works as a developers, providing coaching in Agile practices and coding.


• Works closely with Team Leader.

Empowerment Required: Empowered to make the technical decisions.
Empowered to use best practices and apply both internal and industry standards.

Developer

• Understands and embraces the Agile process, its benefits and core practices


• Is able to produce good quality, well designed code in response to stories


• Participates well within the team, adopts all required practices and attends all sessions.


• Works closely with Tester (QA) on Story


• Recognises the authority and expertise of Technical Lead (Coach) and uses them appropriately.

Web Application Production Line II

The Problems

Design Spec

User Interface Design use a process that is more appropriate to the Waterfall methodology. A lot of up-front design time is spent finalizing page designs, page flow, graphics and copy, running customer workshops and updating design decisions based on their feedback. Then a final spec is produced, which has to be signed-off by many interested parties, before it is handed over to development to implement.

The output from this stage of the process creates problems for the rest of the process for the following reasons:-

• The spec usually takes so long to be finalised, that the development project is usually nearing completion before it is signed off.


• Late changes to spec negatively impact the project just before final testing


• The spec contains static pictures of page designs, but there aren’t any HTML prototypes, so they haven’t taken into account the limitations or features of HTML and CSS.


• There is very limited scope for iterative feedback and prototyping, and every change will have to go through sign-off again, causing more delays.


• When the developers begin their work on the pages, they learn far more about the problem area, and have feedback that can substantially change the page designs.

Brand Guidelines

Brand guidelines aren't always appropriate for today’s internet. They prescribe layout that doesn’t take into account the limitations and features of XHTML and CSS or different devices. Brand can be more appropriate to print media and needs to be less prescriptive and give guidelines for look-and-feel rather than dictating every pixel on the page.

Current Process

Traditionally, the process is as follows:

• Idea is proposed


• User Interface Design produce full page designs on paper


• Graphic designers create all the required images, mock-ups and Flash demos


• Customer testing performed using static page mock-ups and Flash demos


• Copy is created and sent to Legal for sign-off


• Full spec is created with user flow, static screen designs and copy.


• Spec gets signed off by various departments, including legal and brand


• A signed off spec of the screens and copy is handed over to the front-end developers and C# developers to implement.

This is very waterfall, and by the time the developers first see the designs, it is already too late to influence them. When the first XHTML prototypes are produced from the front-end developers, the feedback loop is not taken advantage of, because the spec is carved in stone. What we need to do is make this a more iterative, lightweight inclusive process and not a heavy-weight, inflexible, exclusive process.

I think that we need to recognise that this creative process has to work as something of a production line, but incorporating increasingly inclusive iterative cycles to gather valuable input and feedback as quickly as possible.

In this Web Application Production Line, we need to make sure that every specialist skill is allowed freedom to work to its fullest without being constrained by other parts of the process. We also need to make sure that all the information is included in the feedback loop so that early decisions made with partial information don't constrain the decisions down the line.

Web Application Production Line

The initial idea will come from the business or marketing, or an R&D department. If this idea is too concrete with regard to the final implementation, the rest of the process will be constrained and won't work optimally. Therefore we should limit the expression of these ideas to Outcomes. This gives the development team the freedom to do their jobs effectively.

The next problem is: what skillset is most appropriate for step 2 of the production line? I think that this is User Interface Design. However, currently, the first stage of our process involves a lot of Graphic Design and static pictures of page designs. I don't think that this is appropriate at this stage of the process. Its too early in the process to be finalising look-and-feel, graphics, spending time getting details just right on pictures produced in Fireworks or some other art package. We need wire-frames and storyboards at this stage - no fancy graphics, just basic sketches that capture the fundamental user interactions with the system, the information displayed or captured, and the scenarios that need to be catered for.

We don't want to spend time getting sign-off from all the interested parties, we want the first cut of the user-interactions defined quickly by the people who are skilled in this area. Then they should communicate and discuss their ideas in person as we move on to the next stage of the process, with the expectation that these wireframes and storyboards will grow and evolve and be changed as part of the iterative process, incorporating feedback from users and developers.

One aspect of the signed-off spec that comes out of the Design step in the current process is that all the page designs and mock-ups have been produced in art packages as if they were for paper designs. Hardly any consideration has been given to the fact that this is a page to be displayed on a browser. Is it fixed-width or a liquid layout? How does it expand with the window? Is the design appropriate for the limitations of HTML and CSS? What about accessibility, load-times, degradation between different capabilities of browsers, different screen sizes? The list goes on and on, but the current spec doesn't address any of these points. Despite this, it still gets signed off and passed to the developers.

In my opinion, the next stage of the process needs the front-end specialist developers (XHTML, CSS with appreciation of UID and Graphic Design) involved to quickly get some basic XHTML documents produced, with maybe some quick (standard?) CSS applied to give the page some structure and the right sort of image. At this point, the browser-specific considerations mentioned above will come into focus and these can be discussed with the interaction designers. However, because the wireframes are so basic, they should not constrain the front-end developers, and they should have the freedom to augment the early designs with their input and record their decisions in the form of an XHTML document.

This is where you can start your user testing and initial feedback loop. Real customers might not be appropriate, given the rudimentary nature of the pages at this early stage, but within a web development team, there are many people who can give valuable feedback at this point. The Truth (Business Representative on the team), QAs (testers), Usability and Front-end Developers are all good candidates to try the early pages and give feedback. Quite often, this unearths some fairly basic scenarios that haven’t been catered for in the design which then needs some more work before it can be developed.

You can go round this initial loop until everybody is happy that the interactions are optimal, the basic layout is acceptable, and the XHTML, accessibility and cross-browser compatibility hit the right level. Then this can be "signed off" by the Design team as their requirements spec, and put into source control. This static XHTML could include comments as placeholders for required dynamic content, and all the various XHTML fragments to support all scenarios that can occur in the page so that the ASP.NET developer can easily understand what he needs to do. Everyone should work on this source-controlled XHTML document from this point onwards.

It is at this stage that the Graphic Designers, the Copywriters, Legal, Brand etc can get involved and, using the XHTML document as the master copy of the spec/design, work to build the finished article with the correct Brand, legally compliant text and the final images.

Design Work in the Planning Phase

A sticking point with this production line and iterative implementation is that once the project requirements have been defined, we want the dev team to begins work on the highest priorities as soon as possible. However, if developers have to wait for the Design stage of the process before they start coding, this will cause a delay, and therefore waste. It's obvious that the faster and more efficient you can get the initial Design stage of the process, the less latency and waste you will have in the system. We first need to ensure that the Design stage is focusing on the pages that the developers need to develop first. It sounds obvious, but it doesn’t happen at the moment. Also, cutting down the scope of the Design stage to purely produce wireframes and storyboards will help speed this up. However, if we want everyone to work on the same pages at the same time, how do you get rid of the lag that the Design phase introduces? One approach that appears to work quite well is to create the wireframes, storyboards and XHTML as part of the Planning stage, possibly in parallel with the preceding iteration. Once the XHTML pages are produced, they can be used as a contract between the various areas to base their work on. While the ASP.NET and C# developers are planning and estimating their work, the Design team can be putting together the basics of the screen flow. The rest of the work (CSS, JavaScript, Flash, Graphic Design, ASP.NET, Web Service, Database, Copy, Legal, Testing) can be based on this hub of an XHTML skeleton.

References

• Wireframes


• Wireframes


• HTML Prototypes


• Storyboards


• Interactive Wireframes

Managing the Team

Give the Team Freedom

I just read an interview with Mike Gascoyne, who is the Chief Technology Officer at an F1 team. Couple of extracts that I thought were relevant to managing Dev Teams:

...part of being a good manager is that you leave your guys free to do their jobs

...no matter how much money you put into building the house, if you don't dig the foundations properly, it falls down. So you've got to do the basics right

...if you go in and start changing everything all that will mean is they won't actually finish it, and everything will be half and half

...the key thing is to put the structures in place, get the priorities set