Your Lean Startup

What is ‘Lean Startup’?

Eric Ries started his first year working at startup IMVU with enormous vigor, building a kind of early Metaverse application. About a year later, the early version of their product was pretty far off the mark and saw little traction with users. Eric had an epiphany- he could have spent the last year sitting on a beach and resting and his startup would be in the exact same spot it was now, economically speaking.

While a lot of us at the time (myself included) basically thought this was ‘OK’, just part of the business, he didn’t. He asked some hard, inconvenient questions about the practice of startups and innovation and, fortunately for us, the result was his seminal book ‘The Lean Startup.

The Lean Startup book and movement grew out of  his work on applying the principles from the lean manufacturing movement to the creation of ‘startups’, defined as any new line of business, whether it’s a standalone company or a venture inside an existing company. The goal of lean manufacturing is to eliminate waste, ‘muda’ in Japanese. In this context, a startup is any venture that hasn’t yet validated a ‘product/market fit’, meaning they do NOT yet have a proposition they can reliably sell to a particular type of customer.

The focal point of Lean Startup is the Build-Measure-Lean loop. It looks like it’s supposed to start with Build, but actually you’re supposed to start with Learn so you have high quality ideas that you’re testing. Here’s some questions it can help you answer:

1. What did you do today to guide your venture (or project) to the outcome you want?
2. How will you know if it worked?
3. Does your team have a visible plan that easily allows them to prioritize completing task A vs. task B, C, or D?
4. Do they understand the overall significance of what they’re trying to accomplish?

After reading this tutorial and engaging in the recommended practice you will be able to:

1. Explain when & where Lean Startup is applicable and why it’s useful
2. Take your ideas and structure them into testable hypotheses
3. Design and run effective Lean Startup experiments to test ideas with a minimum of waste
4. Instrument observation into your project’s key outcomes so you and your team can objectively decide what’s working and what isn’t

Why is Lean Startup important?

There’s no simple, straight line process that will reliably beat the intrinsic uncertainty of an innovation project. When I talk with a management team about their innovation program and hear that 100% of their new innovation investments have been successful, I know something is off. A really good early stage VC will succeed at a rate of something like 1/5 to 1/10 of their investments. Established companies have domain knowledge and incumbency they can apply to innovation projects, and most of the one’s I’m met have the potential to do better than VC’s. But it’s wildly improbable anyone investing in innovative new ventures will do 10/10.

To date, there’s only one reliable way to maximize the returns on innovation investments and that’s by minimizing waste- this is where Lean Startup really shines. Rather than promising to make all your innovation investments successful, it offers a reliable way to test new ideas so that one the one hand you don’t miss the next big thing (ex: Blockbuster on mail order DVD’s) but on the other you don’t throw good money after bad (ex: IBM’s AI initiative, Watson).

The particular importance of all this to any innovative company (which is pretty much any growth company) is non-obvious and also breakthrough. Pretty much everything you currently learn about ‘business’ was creating for operating a factory that produces commodity widgets. For such a business, 5 year plans are great, the assumption of perfect information is relatively valid, and your conventional MBA will serve you well. For any innovation-based business, these techniques are grossly inadequate and will generate massive waste.

Who should do Lean Startup?

My short answer is: anyone who cares about whether what the company is offering or developing is actually going to sell.

There’s a pretty specific reason I don’t have a better answer on this: most companies still operate in a legacy paradigm where there’s one department that has product ideas (ex: strategy, marketing, R&D) and another one that takes their wisdom and actualizes it (operations, sales, product management). Most companies don’t have it together to differentiate between their learning/testing work for new ideas and their scaling/optimizing work for existing business that already has product/market fit.

If you’re ready to do things a little differently, this is an opportunity for you.

When should you do Lean Startup?

OK, just as a warning- you’re proceeding to the advanced material on this page. If you just want to know how to describe Lean Startup, you’re probably done. Thank you for visiting the site (and, please, for the love of god, buy my book).

Here, we have to take a minute to unpack what ‘Lean Startup’ means relative to all the other ideas and frameworks out there, some of which are silly, many of which are quite useful for the right job. Let’s assume you operate a line of business with considerations on both cost and revenue. You are building a digital product and operate a product pipeline something like this:

How do you do the best possible work on going from ‘product priorities’ to ‘released product’? You can work harder and faster and generate more output (see: ‘Application Output’) and you can get better at releasing frequently so that new UX doesn’t sit around long before getting to users (see: ‘Continuous Delivery’). And if you’re really at the vanguard, you close the loop with ideas and make evidence-based product decisions (see: ‘Hypothesis Testing’).

But it’s also important to avoid building stuff that no one fundamentally wants, and there’s a lot of that. Consider this: take any bad idea and even with strong practice in the areas I described in the last paragraph, you wouldn’t know you had a bad idea until ‘Hypothesis Testing’, at best. Fortunately, we there exists a portfolio of tools for improving the odds that you are building something the user wants and also linking that assumption forward to your work downstream in the pipeline such that you minimize waste.

To do you do disciplined work in design (‘Continuous Design’) that’s compatible with strong, disciplined work in the rest of the pipeline, I like to use the tried and true ‘double diamond’ framework of design and then overlay hypothesis areas on those:

Before applying the tools of Lean Startup, make sure you know who and what you’re testing with validated Persona and JTBD hypotheses- see step 01 in the ‘How?’ section below on this. From there, your goal is to test demand without building an actual product, a 1.0. After that testing, if you find something worth building, you work on testing usability and testing the user interface, work which is best served by a related but different set of tools- for example, user stories and usability testing.

How do you get started?

Two core practices underlie lean: 1) use of the scientific method and 2) use of small batches. Science has brought us many wonderful things. I personally prefer to expand the Build-Measure-Learn loop into the classic view of the scientific method because I find it’s more robust. You can see that process to the right, and we’ll step through the components in the balance of this section.

The use of small batches is critical. It gives you more shots at a successful outcome, particularly valuable when you’re in a high risk, high uncertainty environment. A great example from Eric Ries’ book is the envelope folding experiment: If you had to stuff 100 envelopes with letters, how would you do it? Would you fold all the sheets of paper and then stuff the envelopes? Or would you fold one sheet of paper, stuff one envelope? It turns out that doing them one by one is vastly more efficient, and that’s just on an operational basis.If you don’t actually know if the envelopes will fit or whether anyone wants them (more analogous to a startup), you’re obviously much better off with the one-by-one approach.

So, how do you do it? In 6 simple (in principal) steps!

1. Start with a strong idea, one where you’ve gone out an done customer strong discovery which is packaged into testable personas and jobs-to-be-done. If you’re familiar with design thinking, it’s very much about doing good work in this area.
2. Structure your idea(s) in a testable format (as hypotheses).
3. Figure out how you’ll prove or disprove these hypotheses with a minimum of time and effort. Much of the time, you can do this without building any actual product. Just to give you a sense of proportion, some of the best ever Lean Startup experiments were run in under a week.
4. Get focused on testing your hypotheses and collecting whatever metrics you’ll use to make a conclusion. Don’t worry about anything else- none of that will likely matter unless you have an idea that proves to be valuable to your customer (or user).
5. Conclude and decide; did you prove out this idea and is it time to throw more resources at it? Or do you need to reformulate and re-test? There’s no shame in the second outcome- the core virtue of Lean Startup is the recognition that startup’s are high risk and it makes sense to avoid waste so you get multiple shots at a winner.
6. Revise or scale/persevere. If you’re pivoting and revising, they key is to make sure you have a strong foundation in customer discovery (see #1) so you can pivot in smart way based on your understanding of the customer/user.

The six sections below describe these steps in more detail. While the focus is on lean and Eric Ries’ work on Lean Startup, a few other techniques (like design thinking and customer personification) are important complements and I’ll reference those as well.

01 Developing High Quality, Testable Ideas

This is the creation story that sells about the founding of new ventures: Young founders dream up brilliant idea, code it, and the next morning are acquired for 1 billion dollars! There’s nothing wrong with a little harmless fantasy, but the reality is that few startups (even successful ones) actually take this course. For most, it’s a marathon of trying things and seeing what works. Did you know Rovio was on the verge of bankruptcy when they released Angry Birds? And that they paid the bills by making games for other companies? I’m not saying doing a startup isn’t fun. It is. I can’t imagine anything better than working with a great team on learning how to build something that matters. But following a fake, media-generated script will probably lead to stress and disappointment and that’s not fun. Let’s talk about how to create strong, actionable ideas.

Your fundamental job is to build empathy for your customer (users and buyers). If you could correlate ‘customer empathy acquired’ against venture success I bet you’d see a very tight correlation. Applying empathy to directed creativity is what the popular rubric of ‘design thinking’ is about. You can learn how to do all this in the customer discovery and personas tutorials.

A good way to make sure you have your bases covered with personas is the Think-See-Feel-Do checklist: What do your persona think, see, feel, do in your area of interest? (Again, more on that in the tutorial above). Following that, you’ll want to frame the value propositions you plan to deliver to the customer in terms of problem scenarios and alternatives. Most of us start with the spark of an idea, ‘Hey wouldn’t it be cool if …’, and that’s fine. But your understanding of the customer will be much more relevant/accurate, actionable, and testable if you can relate the propositions to customer problem scenarios and current alternatives.

02 Creating Testable Hypotheses

If you organize your customer discovery and resulting ideas as we reviewed above, it summarizes naturally into what I call a ‘venture hypothesis’.

VENTURE HYPOTHESIS

‘A certain [Persona(s)] exists…

…and they have certain [Jobs-to-be-Done(s)].

Currently, they’re [using certain Alternatives], but…

…if we [offer our target Value Proposition], then…

…we’ll observe [success through Pivotal Metrics- acquisition, onboarding, engagement, retention, etc.] .’

Here’s an example from Enable Quiz, an example company-
There are HR Managers in charge of recruiting technical talent, 
and they need to screen recruits for the specific technical skills in a job description. 
Currently, they do their best by checking references and asking a few questions, but 
if we offer a way to automate quizzing for a specific job description, then
we’ll observe HR Managers creating and using quizzes and standardizing on use of the platform for new hires.

Most students and advisees I work with find this is a useful jumping off point to formulating hypotheses since it helps to keep their work on personas and problem scenarios linked to their ideas about value and experiments on buyer motivation. The table below organizes key hypotheses areas into into five chunks. (They’re in the form of questions not hypotheses since I think that’s an easier starting point.)

Area	NOTES
Persona Hypothesis	Does this persona exist? Can you name or find 5-10 examples? Can you identify them out in the real world? Do you understand them really well? Do you understand how they relate to your area of interest? Think? See? Feel? Do?
Job-to-be-Done Hypothesis	Do the jobs-to-be-done you’re solving really exist? Is it more of a problem or a need, desire? How important is the problem or problems? How is the customer solving them now? With what alternatives?
Value (Motivation) Hypothesis	How much better than the best alternative is your product at delivering on the problem? How obvious is that to the customer? How will you test that without just asking ‘do you want this?’? (because that doesn’t work)

The areas above are progressive, meaning that you should generally move through them sequentially and that if you’re stuck on one you’ll probably have trouble downstream.

Once you have a working view of where you are and what’s important in those areas, you’ll want to start breaking down them down into individual, testable hypotheses. Let’s pause a minute: This may feel like a lot of stuff, a lot of work, but, remember, this is a systematic way to work through basically everything about your venture. Back to those hypotheses, I like the following format:

FORM OF A HYPOTHESIS 

If we [do something] for [persona], they will [respond in a certain way]. 

That format anchors to the key elements of your work and establishes causality. You’ll also want to consider which, in fact, need proving and how you’ll do that. The only exception to this is that sometimes the Persona and Problem hypotheses may be formulated a little differently- as statements about who the customer is and how well you understand them.

For example purposes let’s use the example of a startup that’s exploring the idea of a talking bicycle compass- it might be a device or just an app on a smartphone. They’ve gone out an interviewed serious cyclists that cycle at least twice per month. They found that the job of finding interesting new routes is complicated by the difficulty of navigating while on a bike. The following table has some examples of how a few of the key hypotheses might look:

Priority	Hypothesis	Needs Proving?	Experimentation
0	Job-to-be-Done: If we ask non-leading questions about how cycling might be even better for regular cyclists, we’ll consistently hear that they wish they could more confidently, more easily try out new routes.	Yes. Sure, the problem probably exists to some degree- but how important is it, really? What are they doing now?	?
0	VALUE: If we offer avid cyclists a way to choose new routes and receive verbal guidance while riding, they’ll buy our [app or device], use it at least once a month, and recommend it to their friend on social media.	Yes, definitely.	?

CLICK HERE FOR HYPOTHESES SECTION OF VENTURE DESIGN TEMPLATE

Next you should determine which hypotheses to test and how to pair them with effective experiments.

03 Designing Effective Experiments

Experimentation is where the rubber hits the road with the practice of Lean Startup. The keys to success are 1) keeping the experiments focused on obtaining a true/false result for one or more of your key hypotheses and 2) being creative about how you can design the fastest, cheapest experiment which delivers on #1.

Not all experiments have a quantitative output- this isn’t physics and it’s valid to review a set of qualitative outputs (like customer discovery interviews) and make a judgement call on how/whether the results prove or disprove a hypothesis. Also, not all product tests should require building product. We’ll go through some quick examples below and more in the MVP Case Studies section, but many of the best Lean Startup success stories didn’t require a single line of code.

Persona Hypotheses

If you don’t know your customer well, it’s time to get to know them (not to mention validate that they exist). When I’m considering an idea, I brainstorm a list of the key personas and then I make sure I can think of 5-10 real world examples. Then you want to make sure you understand them well- What kind of shoes would they wear? Your primary vehicle for validating these hypotheses is customer discovery interviews.

You also want to make sure you understand how they think about your area of interest (using think-see-feel-do). You’re passionate about your idea, but be sure that doesn’t blind or bias you from seeing these people as they really are. Particularly at this early stage, it’s important to be focused on discovering what’s really important to this customer.

For example, even if our team with the talking bicycle compass has some early evidence that this navigation problem exists, it’s probably worth investing a few days to interview more subjects and get more depth and specifics for the personas and problem scenarios. For example, what are the triggers for when they want to explore new routes? Is there a social/multi-rider angle that’s important? Of all the potential cyclists, is there a specific early market that really needs this where they can maximize their odds of getting some early wins?

When I teach Venture Design classes, I usually have students try to write up their personas as a first step- I always find it’s a good way to find out how little I really understand. For your persona hypothesis, this takes the place of the type of hypothesis table you saw above.  You’re looking to create characters for your narrative at this point, not collect statistically significant data, so I recommend against questionnaire. That said, I do recommend prepping an interview guide for focus, consistency, and as a kind of checklist. Quality learning in this area will not only help you stay on the mark, but if you later pivot, it will help you make vastly smarter pivots, steered by the empathy and understanding you have for your customer.

OUTPUT

• Interview guide
• Quality interviews with relevant persons
• Validated personas, including think-see-feel-do

Note: From a process perspective, I would group this item and the next item on problem scenarios, approaching the same set of subjects with a single interview guide. I’ve just divided them up for conceptual clarify (I hope!).

Job-t0-be-Done (JTBD) Hypotheses

Here you’re looking for opportunities- problems/jobs/desires/habits that are important to the persona and where you think you may be able to produce something better than the alternative. There are no new tasks in the enterprise; there are no new consumer behaviors. Do we do things differently than we did 20, 50 years ago? Yes. That said, everything ultimately ties back to an existing need or behavior. Your job is to identify the ‘jobs’ where you have an opportunity to over deliver against current alternatives. This one’s also executed primarily by way of customer discovery interviews.

For example, our bicycle compass team wants to learn more about the jobs/habits/desires around cycling outdoors. They’ve identified the following two problem scenarios:

1: Finding new routes to explore that are a fit with the individuals current interests, location, difficulty, and time available

2: Following said routes

3: Sharing said routes with friends (who also have an affinity for cycling)

But they’re not presupposing they’re right about all (or any) of this- otherwise what’s the point of doing these interviews?. Their job is to get the cyclists to speak freely about what it’s like to go cycling and what, if anything, they wish was better relative to the specific alternatives they use today. A good practice during interviews is to start with questions like:

1. ‘Tell me about the last time you [went cycling].”
2. “What’s hard about [figuring out where you’ll cycle]?”
3. “What are the top 3 things you wish were better about [cycling outings]?”

Don’t lead the witness. These questions are sequenced very purposefully: they progress towards an increasing degree of prompting. They would not want to ask a question like ‘Is it hard to find great find new routes?’, at least not until the very end because even a ‘yes’ provides very little validation. And in many cases you may actually hear ‘no’ for one reason or another where if they really thought about the whole problem, you might get a more useful or actionable answer. What provides a lot of validation is if they ask the first two questions above and consistently hear things like:

– it was nice but I’m getting a little bored of doing the same route

– I tend to stick to the routes I know because I’m worried I won’t get home in time

– It was great- except I got lost!

If they consistently hear responses like that, they can conclude that, yes, they’re on to something. If they don’t, it’s time for caution. This may not be the right job to do, problem to solve.

Once you have a good sense of a few focal problem scenarios, work to understand the customer’s alternatives, how they’re solving the problem, doing the job today. This is an important counterweight to our next area, the value hypothesis.  For the talking bicycle folks, that might be things like: a printed map they marked up, a site they currently use, or a search they did on Google Maps.

Strong presentation of a problem scenario is blood in the water for you- if the current alternatives were really good it’s much less likely you’ll see a strong presentation of the applicable problem scenario. But some problems are just really hard and you need to make sure you understand what a superior (enough) delivery looks like.

Also, you need to make sure the problem/need happens enough. I met a serial entrepreneur who had written a beautiful, functional app for finding movie times. Users loved it. But it didn’t add up commercially because pretty much no one watches enough movies to use or care about the app enough to add up to a good business.

OUTPUT

• (interview guide prep. and interviews per above)
• Validated jobs-to-be-done, including a description of current alternatives

Demand Hypotheses

Armed with real personas and validated problems, now you have to validate whether or not what you’d create is better enough than the alternative to make a sale. Sadly, the way to do this is definitely not asking the customer whether or not they would hypothetically buy your product in the future.

This information is less than worthless because it creates a creates a false validation that we desperately crave for our idea. When it times to actually go out and sell the thing, you may encounter an echoing silence. I say likely because just about everyone will say ‘yes’ to a hypothetical sale- they don’t want you to feel bad and even more so they don’t want to argue with you. See the Yellow Walkman Story for a great example.

If you can’t ask customers, then what are you supposed to do? Just build something and hope for the best?

No! There is a better way: the Minimum Viable Product (MVP). The naming is kind of unfortunate because the point of an MVP is to avoid building an actual product. The idea with the MVP is that it’s the minimum ‘thing’ you can use to test your value hypothesis. Ideally, it’s a fake product of some sort or ‘product proxy’, as I like to call it.

Why? Well, as I mentioned in the introduction, the whole point of lean is to avoid waste. Building something that no one wants (which happens all the time), is exactly the kind of waste Lean Startup seeks to avoid.

An MVP is not the same thing as the 1.0/first version of your product. In fact, this is one question I pose at the beginning of workshops: What’s the difference between an MVP and a 1.0? The basic difference is that the MVP is a vehicle to test your value hypothesis and the 1.0 is a vehicle to execute on and scale a validated (positively tested) value hypothesis.

A lot of the actual practice of Lean Startup revolves around designing experiments (including MVP’s) that are a good fit with your hypothesis and current understanding and then running those experiments to a definitive conclusion (and often iterating through that a few times). The sections below describe a few common patterns or archetypes for these experiments.

The Concierge MVP

The basic idea with a concierge MVP is that you hand create the experience you want the customer/user to have, and see what happens. I mention this one first because it delivers the high quality, deepest observations, and that’s useful when you’re in the early stages of a new concept.

In our talking bicycle compass example, a concierge MVP might be to attach a smartphone to an interested subject’s bicycle, turn on GPS tracking, call them and give them verbal guidance to help them follow a route they’ve selected. A scrappy entrepreneur could, for example, solicit subjects in front of a bicycle store located near a popular cycling location (we have several such venues in the Bay Area where I live).

Lean Startup is very focused on observability and metrics (as is its foundation, science). While the Concierge MVP is more about qualitative observations, there are some specifics you can focus on, even recording them into discrete variables in some cases. This is mostly about observing customer participation and focus/involvement in various steps or branches in the process. For example, with the bicycle example, you might be interested in how they select a route with you or whether they already know their route. You might be specifically interested in whether they follow the route they originally selected. You might also be interested in the social aspect- whether they’re the navigator for a group of friends.

This MVP probably has the lowest value in terms of a definitive result on demand for your proposition, the Sales MVP (see below) being the best. That said, as long as you create the right kind of separation, there’s no reason you can’t add another test to the end of your concierge process. For example, at the end of your concierge MVP you could give cyclists a flyer to sign up for a free beta of your upcoming product and see if they opt in once they leave your presence.

For anything that’s business-to-business (B2B), consulting is often a terrific concierge vehicle- solve the customer’s problem by hand and then identify what can be standardized and automated and build software for it. I love this pattern so much I did a talked for the Lean Startup Circle about it: ‘B2B Hacks- Getting From Consulting to Scalable Products‘.

The Wizard of Oz MVP

The basic idea with a Wizard of Oz (WoO) MVP is to test a target user experience without real working software- there’s a (wo)man behind the curtain making things happen, hence the mention of Oz’s wizard.

Robotics is one space where the fake back end WoO pattern is popular. I was just interviewing a product manager from iRobot for The Interdisciplinarian. They make the (awesome) robot vacuum ‘Roomba’ and often test new feature ideas by having users interact with their robots while a human operator executes the interaction they’d implement in code if the feature looks like a winner.

A WoO test for our cycling example might employ a chatbot and then a filtered voice for navigation- all with real humans actually operating them. A user would download the app and then use the chatbot to evaluate and decide on a route (or just supply it). The nice thing about a WoO is that you can see how a user would actually use a given interface without detailing it out- for example, does the user type responses to the chatbot or draw on a map and post the image back to the chatbot? The balance of the experience would be similar to the Concierge MVP except that now we’re trying to make it seem like the voice responses are programmed (filtering and changing the real person’s voice) instead of just presenting them as coming from a human. Our user is likely to respond somewhat differently in this case- or maybe not, but that’s what testing is about.

Metrics-wise, we might be interested both in the usability of the ‘fake’ software (are they able to/willing to complete a given task) as well as their affinity to interact with it.

The Sales MVP

The basic idea with a Sales MVP (or Smoke Test MVP, as it’s sometimes called) is to see if you can sell something before you actually have it. For example, I would characterize a Kickstarter campaign as a Sales MVP. Running a series of Google AdWords campaigns for your hypothetical product and measuring CTR (click-through rate) is a Lean Startup classic. At the next step in your funnel, bringing visitors to a landing page with a proposition and a call-to-action for an email signup is another classic. In a B2B context, asking for a deposit or funding for a specific development is something that can serve as a Sales MVP.

All of the B2C (business to consumer) options are, of course, possibilities in the bicycle example. The team could test a few pairings of Google AdWord ads and landing page designs/propositions and look at sign-up. Metrics-wise, they’d just be looking at performance along their funnel- click through rates, conversions to email sign-up for visitors to the landing page, etc.

Why not just go straight to the Sales MVP if it’s so good at validating hypotheses? The main reason is that it doesn’t offer much depth of observation. Frequently, teams will observe very low CTR’s and then not be sure where to go next. The right place to go is probably an MVP with more observational depth- the concierge or the Wizard of Oz, for example.

Summary of MVP Archetypes

MVP Archetype	Metrics	Observational Depth	Hypothesis Validation
Concierge	Mostly qualitative- sometimes discrete observations on subject participation/interest in different steps/branches of the experience	High	Low
Wizard of Oz	Subject progress through interface (like a usability test) and affinity for experience	Medium	Medium
Sales	General growth metrics- click through rate, sign up, open and response rate on emails	Low	High

CLICK HERE FOR EXPERIMENTS SECTION OF HYPOTHESIS-DRIVEN DEV TEMPLATE

Customer Creation Hypotheses

Once you’ve got an identifiable customer that reliably/repeatable buys into your proposition, you have the nucleus of your ‘product/market fit’. The last frontier is to make sure you can multiply that product/market fit on an economical, repeatable basis. There are certainly dozens of products out there that I’d buy if I was fully informed about them. But that’s never going to happen. We live in a world of imperfect information and rising above the noise floor is a real challenge.

You’ll want to break that hypothesis down into more manageable pieces, otherwise you risk freaking out and just asking ‘Where the #@$# are all my customers!?’ For this, I like the ‘AIDA(OR)’ framework- Attention, Interest, Desire, Action. Since this framework is over 100 years old, I added Onboarding and Retention, important additional steps for the kind of products we sell today.

In practice, this application of Lean Startup is about running a lot of small growth experiments with a focus on metrics. That said, it’s easy to get lose in those details and I think a shared, qualitative/descriptive view of what you’ve learned about the customer journey is a critical focal point.

I really like storyboarding as a way to walk through the AIDAOR breakdown and keep it connected to your work on customer discovery:

Here’s an example (totally made up by me!) for Home Depot (hardware store):

Here’s the post it came from if you’re interested in the detail: Storyboarding Customer Acquisition.

Now you can start thinking about this journey as the backdrop to your customer funnel:

The contours of your funnel may differ, of course, but the basic idea with this area is to isolate and focus on one area of your funnel at a time. The table below provides focal notes across the AIDAOR breakdown. Note- some of these hypotheses and items will bleed across AIDAOR some. It’s not a hard science. The important thing is to make sure you have a clear, testable view of the customer journey that dovetails (and/or updates) your work on customer discovery and personas.

Step	Sample Hypotheses	Notes
Attention	On AdWords we can achieve click through rates of [x] at a cost of [y]. We can achieve a viral coefficient of [x] on emails. Our salespeople can call on [x] qualified customers/day.	This is where you start, obviously. How does the customer find out you even exist? How do you get them to click through to your site? To sit down with one of your representatives? I hear ‘someone will tell them about it’ and that’s OK but you should have a clear, measurable view of how that word of mouth happens. Generally speaking, your job is to communicate with the customer persona in a way that’s relevant to them and present them their problem scenario in a compelling way, connecting with their understanding of it.
Interest	If we can achieve a bounce rate of [x] on our landing page, then we can achieve a success rate on scheduled meetings from cold calls of [y].	They took a look what what you showed them- are you connecting with a relevant problem scenario? Credibly? Do you have a value proposition that they think will deliver on that problem scenario?There are a few tricks for getting attention, but this is where your work on customer discovery will really bear dividends- the best messaging is crafted around an intimate understanding of the customer. Many great items are also arrived at by iteration so just like everything else here, this is very much a place to A/B/..N test different versions of your proposition, different channels, etc. and learn what works.
Desire	Customers will share out our messages at a rate of [x].We’ll see comments and feedback that we’re really connecting with what the customer wanted to solve their problem.	Remember the ‘Feel’ part of think-see-feel-do in the persona from step 01? This is where that becomes important. Most of us do a lot of what we do for reasons that are ultimately emotional. And you’re competing with a lot of other demands and distractions on your customer’s time. How are you connecting with them?
Action	If we get [customer segment or persona] to a landing page with a demo, [x]% will sign up for [our email product announcements, a free trial].	This is whatever the customer has to do to buy your product. Make sure to keep it as simple as possible. Many companies make the mistake of creating great product and promotion and then having a crummy sign-up process or onerous contracts. How will you know if you’re doing well here?
Onboarding	If [customer segment or persona] signs up for a free trial, at least [x]% will [complete some kind of action on the app].	This is whatever’s required for the customer to a) start really using your product and b) make it a habit (consumer) or integral to their processes (business). How do you review and ensure customer success? This is the last place you want to be
Retention	[x] portion of customers will renew/re-purchase.	How well are you doing on renewals? Up-sells? Word of mouth from existing customers? It’s much easier to work with the customers you already have.

Usability Hypotheses

This area has to do with testing interface patterns for maximum usability. Referencing the Fogg Curve above, this is the Ability/Usability dimension on the x-axis whereas the Value Hypothesis dealt with motivation on the y-axis.

In practice, success here has a lot to do with bringing your validating learning forward for development in the form of user stories.  If you’re not familiar, these are basically a way to be specific and detail oriented about the experience you want the user to have with out prescribing the implementation.

This is important for the practice of lean/Lean UX because it frees you (conceptually) to prototype and test several different approaches. For more on how to do this, see the Customer Discovery Handbook- the section on usability. Key to doing this successfully is getting your crucial assumptions on the board early and making them a focal point for your subsequent exploration and testing.

For example, let’s say you’re considering a web-based drag and drop interface and some on the team think users wont’ get it and others are certain it’s the best option. Get an assumption on the board like ‘If we present a drag and drop interface to [deliver on whatever user story you’re working], 95% of users will engage with this affordance to accomplish their goal’. Then test. With Lean Startup, it’s all about testing!

04 Experimentation

If everything’s in order up to this point, this should be easy! Stay focused on the experiments, get them done, and then move on to a decision about whether to revise or move forward.

That said, nothing ever goes perfectly and distractions arise. The top determinant of successful experimentation (in new ventures) is focus. Creating output makes us feel good- do the work, cross it off the list, call it done. But that’s not how to assure your best possible outcome under uncertainty. With Lean Startup, you have to be ready to cross something off the list and then likely put it back on the last several times before you get it right. Emotionally, it’s daunting.

Here are 3 tips to stay on track:

1. For everything you’re doing on step 03 (designing the experiments), make sure you’ve visualized the moment where you interpret the results and make a decision. If you can’t visualize that moment, you probably need to tighten up your experimentation/discovery plan.
2. If you’re thinking that an experiment isn’t going to deliver a definitive result, odds are you’re right. Stop it, fix it, repeat it.
3. Everyday, ask ‘What did I accomplish yesterday? What will I do today? How did those things tie to the outcome I’m pursuing?’ Here’s a post on a related technique: The Daily Do

The first bit of experimentation deals with customer discovery and validating your idea. I’ve found that’s somewhat unfamiliar territory for a lot of folks, so I put together the checklists below to help you step through that process. These checklists describe a few key items you should verify within the persona and problem hypotheses.

Checklist: Persona Hypothesis

	Hypothesis	Experiment
✔︎	This persona exists (in non-trivial numbers) and you can identify them.	Can you think of 5-10 examples? Can you set up discovery interviews with them? Can you connect with them in the market at large?
✔︎	You understand this persona well.	What kind of shoes do they wear? Are you hearing, seeing the same things across your discovery interviews?
✔︎	Do you understand what they Think in your area of interest?	What do you they mention as important? Difficult? Rewarding? Do they see the work (or habit) as you do? What would they like to do better? To be better?
✔︎	Do you understand what they See in your area of interest?	Where do they get their information? Peers? Publications? How do they decide what’s OK? What’s aspirational
✔︎	How do they Feel about your area of interest?	What are their triggers for this area? Motivations? What rewards do they seek? How do they view past actions?
✔︎	Do you understand what they Do in your area of interest?	What do you actually observe them doing? How can you directly or indirectly validate that’s what they do?

Checklist: Job-to-be-Done Hypothesis

	Hypothesis	Experiment
✔︎	You’ve identified at least one discrete JTBD (habit/need)	Can you describe it in a sentence? Do others get it? Can you identify current alternatives?
✔︎	The JTBD (habit/need) is important	Do subjects mention it unprompted in discovery interviews? Do they respond to solicitation (see also value and customer creation hypotheses)?
✔︎	You understand current alternatives	Have you seen them in action? Do you have ‘artifacts’ (spreadsheets, photos, posts, notes, whiteboard scribbles, screen shots)?

Hopefully those help you focus your thinking and progress on validating those early hypotheses.

Checklist: Demand Hypothesis

Every experiment for a Demand Hypothesis should have the basics you see in the checklist below. For some examples, see Example 1.

	Item	Notes
✔︎	Explicit Hypothesis	Which hypothesis are you testing?
✔︎	Test Design	How is the test going to work? Try to really think through the details and (more importantly) test your experiment as soon as you possibly can and leave yourself time for revision between iterations. It’s not important to collect statistically significant data for most of these so feel free to tweak the experiment if you need.
✔︎	Pivotal Metrics	What specific metrics will you collect as the experiment runs? Do you have the right observation instrumented into the test design to get these?
✔︎	Invalidation Threshold	At what value will you consider the result a positive (validation) vs. a negative (invalidation)? Even if you’re not certain, in practice it turns out that putting a line in the sand here is super important. Otherwise, you may never get to a place where you really are metrics driven (or you’ll get there too late). It sounds weird, but even if you think you might revise them later, I’d start with an explicit threshold in mind.
✔︎	Next Steps	What are you going to do if it’s a positive vs. a negative result? This is really important because the point of all this is to drive toward a good outcome for the venture. If the results of your experiment aren’t moving you forward, the experiment may be a waste.
✔︎	Time & Money	Beyond the obvious purposes of budgeting, it’s good to estimate this early since in many situations you’ll have to make a choice on which experiments you actually run.
✔︎	Iteration Time	How long will it take to get results. This is important for the same reasons as Time and Money.

05 Pivot or Persevere?

I recommend settings goals for your experiments and time-boxing them in agile-type sprints (iterations) of 2-6 weeks. This will help keep everyone on track. If the experiments are running well, you should arrive at a ‘pivot or persevere moment’ where you have the learning to decide whether to proceed or revise and re-test. Or you may find you need to tighten up your experiments and repeat them- that happens.

The hypothesis areas above were organized roughly in sequence and the table below describes common results from these experiments and ideas on how to interpret them and make decisions about what to do next.

Concluding on Your Persona Hypotheses

It’s OK to enter one without set hypotheses. Let’s say you’re generally interested in problem scenarios around 3D printing in the consumer durables vertical. That’s a perfectly OK starting point for some customer discovery, driving to explicit written hypotheses as you learn more.

That said, you’re looking to drive to a relatively conclusive understanding of your key personas before you move too far ahead- otherwise you’ll likely be operating on a weak foundation. Here are a few notes on sample conclusions in this area:

Conclusion	Notes
‘Everyone is my customer!’	Ultimately, this may be true but it’s important to identify an early market where you’ll focus and establish a beachhead.
‘There are a few customers to focus on- I’m not sure which one’.	Take your best guess and choose, but run your experiments against a focal early market. Pick the one that has the most compelling problem scenario.
‘I can’t find anyone to interview’	Then I would step back. This almost certainly means you’ll have trouble with the next steps as well.
‘I think I get this persona, but I’m not sure about the whole think-see-feel-do thing.’	Getting down a solid think-see-feel-do for each key persona will not solve all your problems. But not having a solid understanding of your customer is likely to generate waste downstream, decrease your chances at success, and make pivots less well informed and purposeful. I’d check out this tutorial and increase your comfort level with your personas: Tutorial- Personas & Problem Scenarios.

Concluding on Your Job-t0-be-Done Hypotheses

While it’s often practical to combine the field work on customer discovery in this area with your persona hypothesis, it is important to have a strong footing with your personas before you finalize your problem hypothesis. This is important because ultimately you’re going to need to sell something to these people and you’ll need to be able to identify them. Also, some problems are so spread out among customer segments/personas and occasional that they’re not a strong fit for a new venture. Here are a few notes on sample conclusions in this area:

Conclusion	Notes
‘During customer discovery interviews, the subjects consistently mentioned our job-to-be-done’	Excellent! That’s a good preliminary validation you’re on the right track.
‘We did a questionnaire and >80% of subjects said they wish [our problem area] was better.’	I’d be very cautious about that result- it sounds like you’re leading the subjects. I’d like a lot of things to be better but there are only a small fraction of those that I’d actually dedicate my time and money to improving. I’d try face-to-face or at least phone interviews.
‘I am in this business/I am one of these personas and I know I have this problem- and I’m sure it exists for most others like me.’	While there are many fabled successes where founders build products for themselves, it’s not the most reliable way to succeed with a new venture. Your expertise/experience may blind you to doing good customer discovery with others like you- which is, of course, your actual market. By all means, play to your strengths and use your expertise but be sure to approach the customer discovery work with a fresh and unbiased perspective.
‘Our product doesn’t really address a problem, exactly, so this isn’t relevant for us.’	First, words are faulty instruments- on a business-to-consumer product, this is just as likely to be a ‘need’ or ‘habit’. And fundamentally there are no new habits and there are no new jobs in the workplace. Be very sure you understand the problem(s) or need(s) you’re connecting with before progressing.
‘Our product is so fundamentally novel that there are no current alternatives.’	See above- there’s a lot less novelty in the world than we think, particularly those of us that come from the technology world. Make sure you have a clear view of how your customer’s fulfilling their needs today or you won’t have a good counterweight to determine if and how your value proposition is relevant.
‘We’ve mapped out the alternative and observed or key personas in action with them.’	Excellent! You’re ready to synthesize, tune and test your value proposition!

Concluding on Your Demand Hypotheses

This is where it all starts to come together (or possibly apart!) for a new venture- Is your value proposition better enough than the persona’s alternatives to generate revenue? Here are a few notes on sample conclusions in this area:

Conclusion	Notes
‘Over 80% of the people we asked said they’d buy our product!’	They’re probably not being entirely truthful, or, let’s say ‘accurately predicting their future behavior’. I’d disregard that result. See- The Yellow Walkman Story for more explanation on why.
‘We did a concierge test and [got paid, got asked by the customer when they could buy our product].’	Excellent! You’re on the fast track of iterating to a successful outcome. Time to look at the contours of an actual MVP.
‘We finished our concierge test. They liked it but as a result it was a long way to conclusive.’	Now that you understand the problem area and concierge execution better, do you think you could get paid for the next one? That’s a good follow-on test. You can also try some of the options below. If you continue to see a lukewarm response, go forward to pivot.
‘We made a bunch of pre-release sales, but they’re non-binding.’	It’s OK that they’re non-binding. As long as you made the agreement with a real decision maker (someone who could buy it for real in the future), you’ve got a reasonably good validation of value hypothesis.
‘We couldn’t make any pre-release sales.’	Why not? Were they not that interested? Or wanted to see real product first? If so, how real? If they’re not interested, try some other experiments but that’s a sign that maybe you should pivot. If they wanted to see real product, did you push them to something that was too-binding? Were they ready to sign up for any kind of follow up? If so, good sign; if not, they may not be interested and were just using ‘no real product’ as an excuse. That’s a call you’ll have to make based on your experience with the individuals.
‘We found a few AdWord-landing page combinations that had better than expected click through and conversion rates to email sign-up’s.’	Excellent! That’s a good validation of your value hypothesis and you’re gotten a jump start on your Customer Creation Hypotheses.
‘We tried a few things with AdWords and landing pages, but the results weren’t great.’	What happens when you try the same thing out in the real world? You may just need to learn about your personas, problem scenarios and how to pitch your value proposition. These tests are good for connecting with existing demand but not for fundamentally understanding it. Try spending some time with real prospective customers.

 Concluding on Your Customer Creation Hypotheses

These results are generally easy to interpret- you convert your prospect through the funnel, or you don’t. And then you try something else. If the preceding items are in good shape, this should just be a matter of finding the right channel and tuning your approach. If you’re struggling here, make sure you’ve kept your work on personas, problem scenarios and the nature of your value proposition tightly integrated with your work here (messaging, etc.) and don’t be afraid to loop back to those if you suspect a more fundamental flaw is dampening your conversions.

06.a Pivot!

Pivots vary widely in size and number. Pivots in the area of customer creation and business model are just about  inevitable. The section above described a few common conclusions about experimental results and the possible implication of a pivot. The worst thing you can do is limp along- organizing your experiments into iterations where you set a goal about conclusion will help you avoid that. Strong customer discovery and encapsulation of those outputs in personas and problem scenarios, which we discussed in step 01, is critical as a rudder for your pivots. A strong understanding of the customer will help you pivot much smarter.

06.b Persevere!

You have the core spark of a successful startup! Congratulations. Now it’s time to scale up and steadily improve the recipe you’ve found. I recommend the material here on business model generation and using agile to tie the items we reviewed here to your actual product implementation.

With creativity and focus, it’s not hard to achieve substantial validation and with that the confidence to persevere. The next section, ‘MVP Case Studies’, summarizes a few of my favorite examples.

7 Minimum Viable Product (MVP) Case Studies

Validating your idea doesn’t necessarily require a lot of money or even a lot of time. It does require focus and the design of substantial, relevant contact with prospective customers. The examples that follow range from household names to little known and run the gamut of product categories. My intention with this section is that you’ll be able to find at least one pattern that’s relevant to your situation, sparking ideas on a creative MVP.

Case Study #1: Sprig

I first heard Sprig’s story from the founders at a Lean Startup Circle event. From Sprig you can order a healthy $12USD meal delivered with a few taps on their mobile app. It’s kind of like the Whole Foods deli meets Uber, or, in their words “dinner on demand … prep time is 3 taps … delectable prices’.

It’s run by an experienced Silicon Valley team and wanting to go to approach VC’s with more than a great time and a great idea, they ran a successful validation experiment within a week of pulling together they founding team.

SPRIG SUMMARY

Item	Notes
Persona*	Paula the Professional- health conscious, short on time, moderate to high income, already uses similar services like Uber.
JTBD	Having a nice, healthy dinner with no hassle and at a price the buyer can afford (<=$12).
Alternatives	Going to the store or an expensive, take-out, or a slow delivery service (>20 minutes).
Value Proposition	Get a healthy meal like you would order a cab (on Uber): “Dinner on Demand … Prep Time is 3 Taps … Delectable Prices” (Sprig Home Page)

* This is me interpolating/guessing on an item; not part of the Sprig team’s explanation.

SPRIG MVP & EXPERIMENTATION

Item	Notes
Key Hypothesis	People like Paula exist and rather than prepping their own meals, ordering takeout, or eating out, they’d prefer to easily order a healthy 12 meal that’s delivered in 20 minutes. If Sprig offers Paula a healthy meal like you would order a cab (on Uber), then she would use and reuse the service.
Experiment	Prep. such a meal and delivery ad hoc for one night; post the offer for delivery on Eventbrite; email friends and acquaintances
Validation Criteria	Does a workable portion of the emailed population respond? Do they like the experience?
Result	Strong preliminary validation- good uptake and good customer experiences

Like a lot of the examples that follow, Sprig’s first MVP required no (new/custom) software, little time and little money.

Case Study #2: Dropbox

I’ll assume you know about Dropbox. But you may not have heard the terrific story about how Drew Houston validated the concept in the face of a crowded, confused market and a difficult technical execution.

When Dropbox was in its infancy, many file sharing services existed- they just weren’t all that good and so few people used them. The Dropbox proposition was that a well executed product would achieve large scale market success. Here’s the tricky part: to do this well across even the very big platforms like OSX, Windows, iOS, etc. was a big job and they needed to raise money. VC’s were reluctant to place such a bet on a space with existing competitors that were struggling.

So the Dropbox team did something very creative to validate their proposition- see below.

DROPBOX SUMMARY

Item	Notes
Persona	Tom the Techie- early adopter who works on projects that require swapping a lot of files between a shifting network of collaborators.
JTBD	a) Sharing files between a fluid network of collaborators, particularly if they’re: big or numerous or change a lot. b) Backing up files for recovery if a device fails
Alternatives	Many existing products, but none of them super compelling and widely adopted. Also, custom setup’s which work but are cumbersome to set up and maintain.
Value Proposition	A file sharing service that truly feels transparent to the user across all major platforms- OSX, iOS, Windows, etc.

DROPBOX MVP & EXPERIMENTATION

Item	Notes
Key Hypothesis	People like Tom (and others in the later market) exist and if there was a really nice, easy file sharing service, they’d adopt it. If Dropbox created a file sharing service that truly felt transparent to the user across all major platforms- OSX, iOS, Windows, etc., then a mass market of users would prefer it over the alternatives, subscribe to it and use it over time.
Experiment	Hand craft a demo (without actual working, releasable software); post it; orient the messaging to the early market; promote it and see what happens
Validation Criteria	Substantial traffic on the video and sign-up’s for product information
Result	Strong preliminary validation

One additional thing that’s notable about Dropbox is that the persona I (questionably) described ‘Tom the Techie’ was what they identified as they early market, the first few folks who felt the problem scenario most acutely and would be most reachable with the value proposition. While their video demo wasn’t exclusively tailored for that market, they added inside references for that market.

Case Study #3: Photo-Social Startup

I advised this company through a program at Stanford. They are still in ‘stealth mode’ so rather than going into the details about their product, let’s take a look at the general pattern for photo-social products, products like Instagram that somehow make the photos we take more interesting on social media.

The user has or takes photos. Rather than just posting them to social media (Facebook, Twitter, etc.) they want to do something with them to make them more interesting- tell a story, enhance them visually, something like that. Then they post them and the whole point is the reward of social acclaim, your social network registering their approval with likes, shares, and comments:

When I first started working with this team they had an idea of this type and were in starting software development. We put that on pause and used Lean Startup techniques (as well as design thinking and personification) to spend less time and money and still validate (or invalidate) their concept:

STEALTH PHOTO-SOCIAL STARTUP SUMMARY

Item	Notes
Persona	Existing poster of photos. Personas: Martha the Mom, Pat the Party Planner, Teresa the Teen Social Butterfly
JTBD	Do something interesting with my photos so that my social graph rewards me with interest and acclaim
Alternatives	Manually enhance photos, use alternative enhancers/amplifiers like Instagram
Value Proposition	[This is something users can do with photos that will generate engaging content for their social graph]

STEALTH PHOTO-SOCIAL STARTUP MVP & EXPERIMENTATION

Item	Notes
Key Hypothesis	People like the personas above would like to enhance their photos using our process and if they do this they’ll be rewarded with approval and interest from their social network. If we offers Facebook users a way to [do something novel] with their photos, they will try the service and convert to a paid version of the app.
Experiment	Manually create output of the type the hypothetical app would produce
Validation Criteria	Posts created in this way create strong interested demonstrated by like’s, comments, and shares
Result	An echoing silence- nobody cared. Time to pivot!

The result was a big, echoing silence- no interest. But the team was much better off for having found that out sooner vs. later and now they’re working on a much more promising iteration of their idea.

Case Study #4: Leonid Systems

I started Leonid Systems in 2007 to explore new ideas for back office IT in the hosted communications space. Leonid’s customers are mostly large infrastructure providers, companies like Verizon and Comcast. But Leonid needed to start small, and do it on a bootstrapped basis. So we started out doing consulting, and we used that as a ‘concierge’ vehicle to isolate, learn about, and validate important problem scenarios for our customers.

The specific problem scenarios require industry-specific explanations, so I’ll skip that for now and instead reference this talk I did for the Lean Startup Circle in San Francisco:

Essentially, Leonid went through a series of MVP’s, starting with consulting, to make sure that we were doing things that were relevant for our customer base.  

Case Study #5: Rapid MVP Testing with Paul Howe & Associates

I heard Paul Howe’s story at the Lean Startup Circle (SF). He an a couple of other veterans had a funded startup to explore business-to-consumer (B2C) concepts in search of a winner.

Their approach was very heavy on Lean Startup- get in, test, and then scale it or get out (vs. doing more customer discovery in a given area). While personally I tend to pick a problem area and spend more time learning about it, I think their approach is probably great if you have a lot of different ideas you want to try and you’re good (or make yourself good) at this type of experimentation.

The concept I specifically remember was a service to tell you how much all your ‘stuff’ is worth by looking at your emails and bank/credit card statements. Instead of diving into this fascinating ‘big data’ problem, they did a concierge MVP where they did the searches by hand for a few test customers. Paul Howe sat down and just manually searched their email and bank records to compile a statement of what they had an how much it was worth. The result? An echoing silence, and they moved on to their next idea (with relatively little time & money spent).

PAUL HOWE & CO STARTUP SUMMARY

Item	Notes
Persona	(not sure; their emphasis was heavily weighted toward testing vs. customer discovery)
JTBD	I have a lot of stuff around that I might want to sell and/or I’m just generally curious about how much it’s worth, how much I’ve spent.*
Alternatives	Manually going through credit card statements or receipts.
Value Proposition	It’s interesting and possibly useful to know how much stuff you have.*

* This is me interpolating/guessing on an item; not part of the team’s explanation.

PAUL HOWE & CO MVP & EXPERIMENTATION

Item	Notes
Key Hypothesis	There are certain personas who would like to know how much their stuff is worth. If Paul Howe & Co. offered a service where you could quickly, automatically know how much your stuff is worth, users would engage with such a service in large numbers.
Experiment	Manually create such a ‘statement of your stuff’ and see if the user cares
Validation Criteria	Users demonstrate an interest in the service (not sure how they specifically structured the validation)
Result	An echoing silence- nobody cared. Time to pivot!

They encountered an echoing silence but were imminently ready to move on to their next concept.

Case Study #6: Zappos

Since they got started in 1999, you can say Zappos a pioneer in the current era of lean startups. Their story is wonderful and simple.

Nick Swinmurn had the idea that choosy shoppers wanted better price and selection than they were getting at their local mall. What he did next was a pure Lean Startup: he photographed a whole bunch of shoes and put them for sale online to see if anyone would buy them. They did and the rest is history.

ZAPPOS STARTUP SUMMARY

Item	Notes
Persona	Sam the shoe-hound- knows what he wants but not where to get it.
JTBD	Sam is wants to buy a specific shoe he wants
Alternatives	Local retailers, but they don’t have what he wants and he is wasting time and getting frustrated. Possibly mail order or wait until he’s in a bigger market to go to the store.
Value Proposition	Make the shoe Sam wants accessible online and make sure he has a great experience so he’ll come back and not have to think about where to find the shoe he wants anymore.

ZAPPOS MVP & EXPERIMENTATION

Item	Notes
Key Hypothesis	Sam the shoehound exists and rather than shopping locally or compromising on what he wants he’ll find and want to buy the shoe he really wants online. If we offer him a wide selection of shoes online, he’ll buy them from us.
Experiment	Photograph a bunch of shoes and put them on a simple website. Promote a little and see what happens.
Validation Criteria	Do they come and buy?
Result	Yes, they did.

Is Lean Startup just for startups?

No, not in the sense you probably mean. Eric Ries defines a ‘startup’ as any business (or line of business) that hasn’t yet found a ‘product/market fit’, meaning that it can reliably sell a known proposition to a known customer. If you have a new line of business or product within an established company, Lean Startup’s probably a great fit for you.

Is Lean Startup the answer?

Not to be coy, but it does depend on the question. If your question is ‘How do I manage this venture systematically to a good outcome in the face of uncertainty?’, then yes, Lean Startup will help you get there. As a planning  technique for innovation, I don’t know of anything better.

That said, most innovative ventures have other questions as well, like:

Who is my customer really, and how do I make sure I’m relevant to them? For this I recommend the work around design thinking.

How do I take a holistic look at the business without toiling over a business plan that no one will read? For this, the business model canvas is handy.

How do I think about a new venture start to finish and understand where we are? For this, I like Steve Blank’s work around customer development.

How do I develop great products quickly, and bridge the gap between ‘business’ and ‘engineering’? For this, agile is tried and true.

Lean Startup’s Top 7 Failure Modes and How to Avoid Them

Lean isn’t a passing fad: it’s fundamentally better suited to innovation than most of the prevailing classical/traditional techniques. That said, it’s widespread use in the innovation/startup context is relatively recent and best practices are emerging and sometimes the hype diverges from the reality of what’s practical. I compiled the list below based on my experience advising startup’s and individuals on the use of lean/Lean Startup:

1. No Pivotal Hypotheses

Subscribing to the general idea isn’t enough to make Lean Startup perform for your venture. You have to actually articulate them, prioritize the few that are truly pivotal, write them down, and use them as your focal point. The sections above, starting with ‘01 Developing High Quality, Testable Ideas‘, lay out a systematic approach to doing this.

2. No Focal Point

Once you’ve identified and prioritized your pivotal hypotheses, it’s important that you use that as your focal point and litmus test for everything you do. Output is not the same as driving outcomes in a startup. Crossing things off our list makes us feel good, but is it really driving to that ‘pivot or persevere’ moment? Subject all your activities to that litmus test.

Make sure your hypotheses stays up to date and is highly visible. Google Doc’s isn’t a bad solution. Here’s a Hypothesis Template you can use as a starting point.

3. Remaining Inside the Building

This is a riff on Steve Blank’s famous directive ‘get outside the building!’. Validated learning is the one and only propulsion for driving to decisions and outcomes with Lean Startup. Without meaningful learning and experimentation with real prospective customers, your Lean Startup will be running in place.

For more on how to do this, see section 03 on designing experiments and section 04 on experimenting.

4. Confusing Usability for Motivation

I see so, so many teams over-investing in excellent usability for a product that it later turns out no one is motivated to buy or use. Usability matters, but I would say motivation matters more at the early stages of a project. My favorite tool for thinking about the difference while considering the relationship between the two is the Fogg Curve- see below. On the y-axis is Motivation: how much does the user/buyer want this? On the y-axis is ability/usability- how easy is it for them to buy it/use it?

If you consider a point on the upper-left of this curve, that’s a product that the user is very motivated to use and will use even if their ability is low (the usability is low). If you consider a point on the bottom-right of the curve, that’s a product that the user is not very motivated to use but would use if it’s super-duper easy.

You need to execute well on both, but with a new project nailing motivation is key. Even thought it doesn’t give you a lot of observation, the Sales MVP is the simplest way to test that.

5. Aimless Pivots

Lean Startup helps you make sure you’re not wasting time on an idea that’s not ready to for success. It doesn’t deal directly with how you determine which ideas are highest quality. For this, I highly recommend the use of design thinking techniques, specifically personas, problem scenarios, and value propositions. This material has the added benefit of making sure that if you do have to pivot, you’re doing it with an increasingly better understanding of the target customer. This increases the odds you’ll arrive at a pivot that hits.

The practice of design thinking is tightly integrated into this tutorial. For more on personas, etc. see: Tutorial on Personas, Problem Scenarios.

6. Lack of Purpose & Goals

The world’s a noisy place. Distractions will walk in the door every day. Many teams with good intent and an understanding of Lean Startup fail to make steady, reliable progress towards a pivot or persevere moment.

It’s important to work in time-boxed (time-constrained) iterations, each of which have discrete goals. That’s what the material on Startup Sprints is about, though there are many ways to implement the concept. The Daily Do is another technique you can use to make sure you and your team are on track day to day.

7. Too Big an MVP

We love to build things, it’s in our nature. Subordinating your love for the product your building to the learning mission at the core of a Lean Startup is difficult (at least, I’ve never found it easy). Doing so requires discipline focus, and clear check points to make sure you’re on track.

The MVP case studies here are a useful test point or you to step through whether or not you’re building too much product.

Please Note: This list presupposes you want to and should use lean to solve your problem at hand. For a view on where lean’s a good fit, see the section above ‘Is Lean Startup the answer?’.

Criticism & Context

In practice, lean isn’t always the right method- a statement you could make about pretty much any method. That said, as a pure idea, it’s pretty durable and coherent. Every method should be subjected to scrutiny and, of course, rigorous validation is itself part of the method. Below are a few summarized criticisms of lean and Lean Startup along with notes.

You can’t skimp your way to greatness.

While this observation may often be true, its application to Lean Startup is mostly the result of misunderstanding. You pair the words ‘lean’ and ‘startup’ and the idea of avoiding waste and it’s not shocking that on a quick look you come away with the idea that it’s about making sure your startup/venture doesn’t spend much money. Keeping your spend down may be an outcome you get with Lean Startup, but the method itself about waste avoidance not cost avoidance.

Earlier, we looked at the Minimum Viable Product (MVP) concept. That fits into a process where you create a tightly defined value proposition, then conceive the most quickest, least expensive way to test it (the balance between cost and speed being mostly a function of your particular priorities). You may reach a point where a relatively long, expensive creating of something is the best way to do that.

Lean Startup wouldn’t say that’s wrong. It would just say that you should exhaust the quicker, cheaper alternatives to testing the proposition so you don’t go through a long, expensive creation cycle and then encounter an echoing silence where customers aren’t interested in what you created.

It doesn’t work in [medical, industrial equipment, other areas with long design cycles].

Sure, there may not be any shortcuts to getting a regulatory approval for a new drug or device. Yes, it may take a long time to get a new model of bulldozer functional. These aren’t good reasons to discard the method.

First, you may be able to test the demand for your proposition without a product. Let’s say you hold a webinar, conference about a particular problem area for medical clinicians- Do a lot of them show up for problem A vs. problem B?

Additionally, there are a lot of elements to a successful customer experience that surround a core product. How do clinicians identify when and how they should use this new product? Buy it? Store it? Take it out of the box and administer it? These are areas where small batch experimentation may be perfectly viable.

For a set of actual examples of how this works, here’s an article about the application of Lean Startup at GE: HBR Article on Lean Startup at GE.

It hasn’t been statistically validated that Lean Startup actually makes companies more successful.  

This is true but not necessarily that relevant for two reasons. First, it’s difficult and rare for social science to reliably draw these kind of conclusions. Success factors for products and ventures vary across a lot of dimensions which change over time with their operating environment. Second, Lean Startup has only been around since 2011- there just isn’t a lot of data available.

I don’t want to oversell lean or Lean Startup, but I do think the criticisms above are mostly the result of misunderstanding or inappropriate context. In practice, I think the biggest issues with it mostly have to do with a) not actually grinding through the details of its rigorous implementation and b) wanting it to be the one silver bullet for every problem and situation (which is natural- who doesn’t want that), when in practice it’s a portfolio of methods that lean to successful innovation.

How Do I Get Started?

In my teaching and advising, I find that these seven steps are an effective way to initiate your practice of Lean Startup:

1. Draft a positioning statement so you have a clear working definition of the project
2. Charter the project with a core value hypothesis
3. Draft three ideas on MVP’s to test your core value hypothesis
4. Storyboard the customer journey with AIDAOR
5. Unpack your hypotheses in more detail
6. Choose what experiment you want to run and complete a working version 
7. Start experimenting!

The first six steps will probably take you 60-90 minutes and they will get you rolling.

1. Drafting a Positioning Statement

We all think our idea is perfectly clear, but we’re often wrong. I really like the standard positioning statement a way to make sure I’ve identified the basics of what the project is about. Here’s the format I use:

For (target customer) who (statement of the need or opportunity), the (product name) is a (product category) that (statement of key benefit – that is, compelling reason to buy). Unlike (primary competitive alternative), our product (statement of primary differentiation).

You can find an example in the sections below (Example 1, etc.) and if you’re using the Hypothesis-Driven Dev template (Google Doc), you can add it here: Positioning Statement in the Venture Design Template.

2. Chartering the Project with a Core Value Hypothesis

Like the positioning statement, this kind of addresses the ‘dumb question’ of whether there’s a clear, actionable view of what the project should do. This has the same form as a ‘regular’ hypothesis:

If we [do something] for [persona], they will [respond in a certain way].

The only difference is that it’s a kind of summary of what the venture hopes will happen. For example, for the startup Enable Quiz that wants to sell a SaaS quizzing solution to companies, it might be something like this:

‘If Enable Quiz offers an easy, affordable, lightweight technical quizzing solution, we can acquire, retain, and monetize customers.’

Simple right? Even though it sounds simple, I recommend doing this to anchor and communicate your core objective.

3. Drafting Three MVP Ideas

I wouldn’t say Lean Startup has no value if you don’t run experiments (I think it’s still good for general focus), but it is mostly about running experiments. Given that, I recommended looping back to experimentation a lot. After you have a core value hypothesis, I’d sketch out what the three MVP patterns might look like to test it:

1. Concierge MVP. How could you create the experience & outcome you want the customer to have with a minimum of up-front work? Remember, practicality/scalability does not matter! You’re only trying to do this a few times.

2. Wizard of Oz MVP. How can you fake the product experience itself? What are you looking for (in terms of motivation)? Very frequently these are paired with a Sales MVP in a ‘show them the Wizard of Oz MVP and have a call to action with the Sales MVP’. If so, it’s OK to reference the Sales MVP for most of your outcome observations and metrics.

3. Sales MVP. How would you find your target customer and pitch them? What’s the pitch?

4. Storyboarding the Customer Journey

You’ve got the big stuff in order- what about the specifics of what might make this experience a go vs. a no-go for the user? For this I really like to storyboard a take on the customer journey: attention, interest, desire, action, onboarding and retention. For more on this see the tutorial on storyboarding (the part on user acquisition) and the example below.

Take your core/summary value hypothesis and decompose it into a more detailed set of at least five ‘child’ hypotheses to test. Move through the AIDAOR process and write down all the assumptions that present themselves to you, using the same formula (but for your more detailed/specific hypotheses): If we [do something] for [persona], they will [respond a certain way]. using the template (attached).

5. Unpacking Your Hypotheses

The Core Value Hypothesis is great for overall focus, but for actual testing and execution you’ll probably want more detail. The good news is that the AIDAOR storyboard you did will probably help a lot for driving to specifics. See below for an example.

Example 1: Initiating Lean Startup at a … Startup

This page presents a set of example hypotheses based on a fictional company, ‘Enable Quiz’. Enable Quiz is rolling out a lightweight technical quizzing solution; for companies that hire engineers, it will allows them to better screen job candidates and assess their internal talent for skills development: Example Practice of Lean Startup at Enable Quiz (Startup).

If you’re doing the case study ‘An MVP, Not a 1.0‘, don’t visit that page until after you’ve engaged in the applicable class session.

Example 2: Initiating Lean Startup in an Enterprise Project

COMING SOON