I first heard the story of the yellow Walkman in the winter of 1999, just as the dot.com period was cresting. It was told to me by my boss at the time, a product design guy and IDEO alumni, as I was walking out the door to the San Francisco subway system to do what would now commonly be called “customer development”. The story of the yellow Walkman is a cautionary tale, though I nevertheless went on to make most of the mistakes this post cautions you to avoid (not to worry, dear reader- I added some surefire tips at the end). The story goes like this:
Sony’s conducting a focus group for a yellow ‘sport’ Walkman. After assembling their ‘man/woman on the street’ contingent, they ask them ‘Hey, how do you like this yellow Walkman?’ The reception’s great. ‘I love that yellow Walkman- it’s so sporty!’ ‘Man, would I rather I have a sweet yellow Walkman instead of a boring old black one.’ While everyone’s clinking glasses, someone had the insight to offer the participants a Walkman on their way out. They can choose either the traditional black edition or the sporty new yellow edition- there are two piles of Walkman’s on two tables on the way out. Everyone takes a black Walkman. [1]
What?! Why would all those focus group participants lie to the moderator like that? There are two reasons and they’re both rooted in the fact that people want to be nice. First, no one wanted to tell they moderator they didn’t like his (or her) yellow Walkman- that’d be mean. Second, group think- it takes a lot of energy and commitment, much more than people have in a focus group, to disagree with the sentiment of a crowd. The moral of the story is that you need natural behavior for quality observational learning. Whoever offered the Walkman’s nailed that.
Recurring Pitfalls in Customer Development
I heard this story almost fifteen years ago and while there’s a definite uptick on awareness about design and design thinking, not a whole lot has changed (yet) in aggregate. As a manager and a startup mentor, I spend a lot of time coaching current and prospective product developers about the practice of design thinking. I do the best I can. I tend to find two opposing misinterpretations about product design.
#1 I call the ‘Steve Jobs Misinterpretation’- see left side of the seesaw below. The basic idea is that that the designer will be like Steve Jobs and Steve Jobs magically knew what was right and didn’t countenance doubt or discussion. The fallacy here is that while Steve Jobs was highly confident in his methods and wasn’t known for taking the long way around the barn, he had an intense, almost manic, interest in learning about the customer. I’m no expert on Steve Jobs but I think that much is clear from the written record.
#2 I call the ‘Tell Me Fallacy’. The basic idea is that being customer driven means having the customer tell you exactly what they want. Think you’ve got a good idea? See what the customer thinks. The problem is that people aren’t able to predict themselves in that way and even if they were, they’ll err on the side of telling you what you want to hear anyhow- see the Walkman story.
OK, so what then? Well, the right approach is a little more subtle but certainly not the proverbial rocket science. It takes a lot of practice to do well, but the basic ideas are easy to grasp.
Best Practices in Customer Development
Step one is to determine what you’re after. There’s discovery and validation. You can do both, but if you do I’d do them in that order. With discovery, you’re looking to learn what you ‘know you don’t know’ and even things about your target user that you ‘don’t know you don’t know’. With validation, you have one or (ideally) multiple alternative product implementations and you want to observe how users react to them. If you’re not sure if you should be doing discovery or validation, odds are discovery’s the place to focus.
If you’re doing discovery, it’s still useful to have hypothesized a few items you want to prove or disprove, even if they’re highly abstract. Without that, you may struggle to focus the discussion. That said, in discovery you’re not looking to fill out a questionnaire, you’re not looking to get a statistically significant result. I’ve had this awkward discussion with startup mentees many times: ‘We talked to people in our target market and 87% said they’d want something like this.’ The yellow Walkman story helps but this discussion’s always rough- they’re disappointed. They’ve done what they’d thought should be done- go out and talk to customers. The problem is that the ‘87% stat’ is probably yellow Walkman data. When you’re in this phase, it’s important to stick to facts and actuals. Every need and emotion your product might fulfill already exists. You need just need to survey the landscape. Your real terra incognita is where your idea can take root and flourish.
You can ask the subject ‘How many times did you eat potatoes last month?’ and expect a good answer. You can even ask them something more abstract like ‘How do you feel when you visit the dentist?’ or ‘Where do you want to be career-wise in five years and why?’. But if you ask them ‘Would you like an app that does [xyz]?’ Or ‘What would you want in an app that does [xyz]?’ then you’re probably going to get yellow Walkman data, and derail the discussion. Particularly if you’re at this early stage, I highly recommend against group discussions. There’s a lot of quality research that even with internal company discussions, these formats devolve into a question of who talks the loudest and most frequently. Keep the discussion natural but focused on your objectives. You’re not looking for the expected thing- you’re looking for the thing you don’t know yet. Be curious, be friendly.
If you’d like more background, I have a few talks about the integration of design thinking, Lean Startup, customer development, business model canvas, and agile here on the site’s SPEAKING PAGE. If you want a starting point, you can check out these notes and this template (downloadable Google Doc) on the application of these principals: CUSTOMER DISCOVERY TEMPLATE. For sure, I have no monopoly on these ideas and there are a lot of great resources online.
If you’re doing validation, that means you’ve worked from a notion of your personas, problem scenarios and stories to a set of prototypes. Your job is to replicate the natural reaction of your target market to these prototypes to decide if and how you should revise. The yellow Walkman situation was an example of validation. Keep validation simple, keep it natural, and, above all, keep it specific. What would be the modern, Lean Startup take on the yellow Walkman situation? I would say it would look something like this: Sony’s product designer mock’s up several colors of Walkman and puts together some kind of an ordering page with the options. Focus group subjects (or just online visitors) are allowed to pre-order what they want. This gets you the same result without having to actually produce a whole bunch of yellow (or whatever) Walkman’s. Yes, you need to figure out what to do with the subject who selects the purple Walkman you never produce, but you’ve got several options on that.
A full review of usability and A/B testing is beyond what you’ll want to read in this post. Suffice to say there’s an abundance of tools and editorial online.
Talk Back
What’s been your experience? How have you handled discovery? Validation? What resources did you find most helpful? What would you like to have that you don’t have?
1. This is a ‘story’ in the sense that it’s a kind of product design urban legend- folks I know in product design know the story. No one knows if it’s really true. Everyone’s had similar experiences many times over, but not as vivid or tellable. The quotes, the clinking of glasses are my own embellishment to the story.
