Five design processes that don’t work

See update at bottom.
Having a “design process” for Web projects is an appealing idea. It codifies principles that designers like into rationales for involving design in major decisions. Since design is imagining, a lot has been written about doing innovation through design. Since I’m getting really old now I have seen many different attempts at making an innovative design process. I can say right up front that the only thing that actually works is working with a designer on a small team that is driven, wants to create something new, and has the courage to go where that leads. As a former advocate and practitioner (for years) of these design processes, it’s my experience that they contribute little of value to the success or failure of a design.
1. User and market research doesn’t work. As Douglas Rushkoff sez:

“…because you can only research the market’s past, not its future, consumer research doesn’t ever lead to true innovation. It only helps companies to sort out some of last year’s trends in order to create an illusion of sales predictability.” — pg 235, Get Back in the Box (Thanks to Bob Baxley for this quote!)

Users (i.e. people) simply will not tell you what new thing they actually want. In focus groups or other artificial situations, people will reject unfamiliar things. Lab studies and interviews can only evaluate performance of elements of a product, not the whole. Live A-B testing has a homogenizing effect on products, reducing them to whatever is immediately familiar to users. Innovation is ultimately about changing user behavior, not hewing to it.
2. Brainstorming doesn’t work. Creation of many ideas and scenarios divorced from reality is good, but limiting this to a two-hour meeting every so often insures that ideation is not part of major decisions. Occasional brainstorming with project teams is often used as a team-building exercise, so everyone is happy when the ideas they produce are close-in, not new or disruptive directions. Make brainstorming part of making the product, not a separate meeting.
3. “Wizard of Oz” videos don’t work. Fleshing-out an idea in a speculative (i.e. faked) animated scenario is a tactic used by some designers to sell an idea that has no existing engineering or business basis. Without constraints, real innovation does not happen, just out-of-control “idearrhea.” Actual innovation is build-able in some way, anything else should be taken as suspect.
4. Personae don’t work. Defining users as archetypes and attempting to see the product through their eyes has some use for avoiding massive disaster, especially for non-designers. But they are of limited use for designers making hard decisions. Personae should not be over-sold as the answer to making the product user-centric, that has to happen in other ways.
5. “Agile” methodology doesn’t work. Scrum is a huge success in helping large companies redefine engineering to give ownership of products to teams and to deliver better products. But the process fails when applied to design and ideation. A two or three-week sprint is simply not enough time to try a few design directions and prototype working code (and I have tried on at least a dozen occasions). Any kind of exploration has to go out the window. A very experienced Scrum master and advocate I’ve worked with agrees that Scrum is best for delivering against a well-defined specification, and even then produces huge amounts of stress on designers (who have to stay somewhat ahead of the other team members in their deliverables).
A good objection to all of the above would be that IDEO and other great companies have been very successful at innovating through design and have many solid methods they use that they detail in books, etc. I would point out that most of the examples people use for success are physical products (like, as is mentioned again and again, the iPod), not Web sites. I think this is because it is much, much easier to create a prototype of a physical product and see how it feels to use it that to do the equivalent with Web work, where application pieces, data structures, and lots of other very abstract design work needs to be done before you can see how something feels in use.
Another good objection would be that for utilitarian, task-oriented projects, the goals are clear enough that all of these techniques are valuable, and that’s true enough. But these kinds of commodity projects (e.g. building a new shopping cart or a content management system) are not what most designers can contribute to in valuable ways.
Creating products is not a rational process; much depends on tiny aspects, timing, and emotional dynamics between people. Designers should embrace this and stop trying to define their value through processes. Designers should ideate through building, directly with engineers on small teams.
Update: Thanks to Kai Turner who made several valuable points, I’ve edited the post to not sound old and cynical quite as much. I should also make the disclaimer that this is a polemic and deliberately foamy-at-the-mouth.