The tinkerer: Google
At Google, the algorithm knows all. Their view is that data and computers are the best way to approach product design.
If a computer can do it, it is probably the best approach seems embedded in the thinking behind their product decisions, often resulting in products that appear bland but are used by millions of people.
A reliance on A/B testing
The net-net of this approach is a strong belief in A/B testing, where two versions of a page are shown to users, each with a very small set of changes. The one that results in the desired action ends up being the champion, moving on to the next round against another version, repeating the cycle time and time again. So, at Google, decisions are not made based on what the product manager or people at Google think but rather as a result of extensive testing. For example, an article in the New York times pointed out:
a product manager tested a different color with users and found they were more likely to click on the toolbar if it was painted a greener shade.
As trivial as color choices might seem, clicks are a key part of Google’s revenue stream, and anything that enhances clicks means more money. Mr. Divine’s team resisted the greener hue, so Ms. Mayer split the difference by choosing a shade halfway between those of the two camps.
Her decision was diplomatic, but it also amounted to relying on her gut rather than research. Since then, she said, she has asked her team to test the 41 gradations between the competing blues to see which ones consumers might prefer.
That type of extensive testing may be good practice (and it seems that it’s work well for Google to date) but it’s not the type of climate everyone necessarily likes.
Trust the data
In a way, Google’s trust in the data given to them by their consumers can be seen as a large influence in the way companies now design games. The whole social games movement, for example, is largely based on data mining and an understanding of interaction metrics.
Today, Zynga is arguably the most successful producer of social games in the world, thanks in large part to the initial success of Farmville. Zynga’s chief designer, Bryan Reynolds, recently explained how the company was using a data-driven approach to its design:
Zynga doesn’t rely on gut instinct to zero in on what users really want. Reynolds said Zynga follows an array of real-time metrics in order to find out what players like, and what they don’t.
One example was of a screen from FarmVille that promoted another one of Zynga’s games, PetVille. The font used in the promotion was originally red. By experimenting with other colors, the studio found that pink fonts, strangely, generated an exponentially higher click-through count than colors including purple, green, and red. Without metrics, Zynga would have never known that.
“Using the data mining, the metrics, you are able to learn the things that are counter-intuitive,” said Reynolds.
The impact can also be felt across most startups today as investors now require you to track and hit specific metrics. In a lot of ways, this is a saner approach to investment than what we witnessed in the 90s (when companies were getting funded based on potential exit instead of business basics) so one can’t complain about the concept but I would venture that balancing out metrics with other forms of input is also important in order to establish the best long-term strategy.