Idea selection methods: balancing efficiency and accuracy

A couple of weeks back I witnessed the fastest idea review session in my consulting career. In less than an hour 50+ ideas were reviewed and put into four categories: (1) needs more study (2) go for pilot (3) implement and (4) dropped for now. That meant each idea got roughly a minute’s attention from the reviewers. Isn’t that too short? How accurate can such a review be? Well, that depends upon whether accuracy was the main criteria in the review. This being the first round of idea review, efficiency was given a priority over accuracy. In this article let’s look at different ways of doing reviews that are either efficient or accurate.

Efficient reviews

Review by walk-around & sticker-voting: How did the speedy review mentioned above take place? 50 sheets with one idea per sheet were spread on a long oval boardroom table. 3 CXOs – CEO, CFO & CTO – walked around the table reading the ideas. They were given coloured sticky dots – one colour indicating each type of decision mentioned above (e.g. go for pilot). Idea authors were present in the room while this was going on. If any of the reviewers wanted to ask a question to the idea author, the discussion happened right there and a decision was taken. Finally, for each idea that was going for pilot or implementation, a champion was identified. Imagine if each of the 50 ideas were given 10 minutes for presentation. It would have easily taken a whole day.

2-1-0 method of review: Terwiesch and Ulrich present a variant of the above approach in their book “Innovation Tournaments” called 2-1-0 review. Two minutes per presentation, one slide or poster per idea and zero questions. Reviewers vote and select the best ideas. With 2-1-0 method you can review 50 ideas in a couple of hours.

With the advent of social networking / collaboration tools, it is easier to do reviews online. For example, the Starbucks idea site shows how each idea gets comments from various other people.

Now, let’s look at a few methods of reviewing ideas where accuracy is more important than efficiency.

Accurate reviews:

Science fair-style review: A couple of months back I got an opportunity to be a review panellist in Titan’s Innovation Bazaar. This was an annual event where 75 project teams were given a booth each to present their idea. Team in each booth had put up prototypes, posters, videos to explain the idea and its benefits. I along with a colleague was responsible for reviewing 25 of these 75 ideas. We had a criteria consisting of 10 parameters on which each innovation project was to be assessed. It took all of us an entire day to assess and shortlist the top ideas. The accuracy of this review is higher than the walk-around-sticker-voting review. However, it takes that much more time.

Real-win-worth it: Another criterion of idea review which I like is called “Real-Win-Worth it”. The name of the criteria tells us what it is about and that makes it easy to remember. See the picture below. Another thing I like about this criterion is that it can be applied progressively through the life of the idea. For example, in the early phase, we can just focus on the “Real” part. Some prototypes and customer feedback would give us some idea on this. Once we are convinced of that, we can focus on “Win” part etc.

To summarize, before planning for an idea / innovation review, we need to decide whether the focus is primarily efficiency or accuracy. There are multiple options available to fulfil either of these needs.