Table of Contents
A Season of Change & Disruption
“HVAC in a Hurry” (HinH) is now itself in a hurry. After decades of happy existence as a prosperous regional supplier of heating ventilation, & air conditioning (HVAC) services for commercial properties, the industry is consolidating. The CEO, Mary Condor, feels they need to either scale up, combine with another firm, or gradually be out-competed into insolvency. Scaling up is her first choice.
She’s brought in Frangelico DeWitt to lead a ‘digital transformation’ program. Mary noticed that her largest competitor has an app their field technicians use. Her original charter for Frangelico was to build an app. Worried that an app might not be the right place to start, Frangelico worked with Mary to decouple the implementation of a mobile app from his charter. His new charter is to use digital solutions to standardize and automate the company’s learned best practices with regular reporting to Mary about what he’s learned, what he plans to do next, and how he’ll assess those next steps.
Solving the Right Problem
Interviewing HVAC Tech’s
Armed with a charter that’s focused but not overly prescriptive, Frangelico’s excited to get to work. Tim, one of the developers on his team, just started two weeks before Frangelico. At his last job, he worked on a content management system and is excited about the idea of centralizing all the third party vendor documentation the technicians use- manuals from various companies that make HVAC equipment. He’s been tinkering with such a solution and thinks he can have something working in a week or so if the team decides that’s where they’d like to focus.
While Frangelico’s first priority is scoring quick wins, he’s worried about investing his team’s time in something the users might not find valuable (and hence not use or use grudgingly). As excited as the team may be to start coding, releasing a working software application is never quite as simple as it looks. Also, team confidence and morale (particularly in Frangelico) will be particularly sensitive to early wins or losses.
On his second day, Frangelico convenes an informal get together to discuss the team’s charter and their next steps. Not surprisingly, Tim is highly vocal about his ideas for a content management system that will organize technical documentation for the field technicians. Frangelico poses the question of what the team has learned about the work of the field tech’s and their use of the current alternatives for finding documentation. After a hodgepodge of comments, it’s pretty clear to everyone that they don’t know much.
Much to Frangelico’s relief, one of the other engineers, Srini, suggests they spend one week doing a design sprint to learn about the work of the technicians. Some of the team question whether it’s a good use of their skills as specialists, but come around to the idea that they’ll work much better together knowing about their user.
The team splits into pairs and collectively interviews 25 technicians using the interview guide in Exhibit B. The idea with this guide is for the team to ask non-leading questions of the technicians and just hear what they have to say. For example, rather than starting with a question like ‘Would you like it if we built a tool to get vendor documentation?’, they might ask ‘What are the top three hardest things about completing repairs?’.
Deciding on the Right Job/Problem to Solve
On Friday, the team gets together to organize their interview transcripts, analyze their findings, and decide what’s next. The output they want to create is a persona and a set of problem scenarios or ‘jobs-to-be-done’. A persona is a humanized view of your customer or user with a focal view of how they relate to your area of interest. Problem scenarios, or jobs-to-be-done, are the fundamental components of what your persona does in your area of interest, regardless of the solution they’re using at the moment. A famous example of this is that when someone buys a 1/4″ drill-bit, what they’re really buying is a solution to the job of creating a 1/4″ hole.
One interesting finding by the team was that no technician mentioned getting documentation as a hard or annoying part of the job. When asked specifically about how they find documentation at the end of the interview, they all mentioned that they had no problem Google’ing their way to the correct documentation on their smart phone. Rather than feeling they were somehow ‘wrong’ about the documentation idea, this finding gave the team creative confidence that this approach of ‘getting outside the building’ really works.
Getting replacement parts to a job site emerges as a favorite job-to-be-done for the team to focus on for a possible software solution. It’s substantial, but not too large. Being able to order a part online (vs. through the central office) looks pretty tractable with software, at least at first glance. Getting all the info for a job prior to arriving on site was a close second in terms of importance, but the interactions are more complex and varied than getting replacement parts and so at first glance it’s a less obvious candidate for an early win.
The table below describes a few problem scenarios/JTBD for Trent the technician.
| Job-t0-be-Done/Problem Scenario | Current Alternative(s) | Value Proposition |
| Carry printed manuals (They think- but later they find out the tech’s are actually Googe’ing vendor documentation and that works fine) |
(They were able to eliminate this VP with customer discovery- the Alternative actually works quite well.) |
|
| Getting replacement parts to a job site. | Call the office and request the part then wait for an update on the phone or through a call-back | We’ll create a self-service parts ordering process with greater transparency on cost and turnaround time, lowering overhead for fulfilling a parts orders and improving customer satisfaction. |
| Getting all the necessary information to arrive at a job fully prepared. | What the customer tells dispatch isn’t always conveyed or consumed by the technician and the customer ends up repeating themselves or the technician ends up with less information than they need
Trent often calls dispatch on his way in and try to get a quick briefing; he reads the notes on the job if they’re available. |
We’ll create a more structured, automated request and dispatch process with better routing and storage of notes from the customer |
Now that the team’s found a problem/job worth solving, they turn their attention to creating a valuable solution, one that’s better enough than the alternative to drive action and materially better outcomes for the user.
Building the Right Solution
Exploratory Testing with an MVP
Central to modern innovation and design practices is the idea of learning before scaling, often expressed as ‘going from 0 to 1’. A key method from the practice of Lean Startup is the ‘minimum viable product’, abbreviated as MVP. Rather than a 1.0 of your first software application, the MVP is a product proxy that you can use for exploratory testing of your hypothetical solution before you invest in software.
The team decides to use a ‘concierge MVP’ to learn more about the process of getting parts to a job site, before they try to standardize and automate the process. In a concierge MVP, the team hand-creates the experience they want the user to have. In the HinH team’s concierge MVP, they individually shadow technicians and handle any parts ordering the technician requires. This gives them a first-hand, on the ground, understanding of how the technicians go from diagnosis to completed repair where a replacement part is involved. The team learns just how cumbersome it for a tech to get a replacement part, a lot of details about the process, and validation that the technicians are able to complete their work quicker and with a better customer interface when they don’t have to handle the admin. side of parts ordering.
Prototyping and Testing User Stories
The output of this activity is a set of storyboards and user stories about the user experience they want to deliver for the technicians via software. The team starts at the beginning with the experience of diagnosing the need for a replacement and finding its pricing and availability so the tech can work out their next steps with the customer to complete the repair. A broad, ‘epic’ user story that describes this might be something like:
‘As Ted the HVAC technician,
I want to know the pricing and availability of a part that needs replacing,
so I can decide my next steps.’
As a sentence, this makes intuitive sense. As a user story, it also has a specific format and that format has a few specific objectives. The format is:
(1) ‘As a [persona],
(2) I want to [do something]
(3) so I can [realize a reward].’
The first clause helps us clearly identify our target user. The second clause is pretty self-explanatory but should be specify some tangible action the user is going to take with the hypothetical software. The third clause is where we instrument for testability: the ‘reward’ should be some testable outcome where can specific observe whether a subject testing the software has realized the target reward or not.
After drafting the working ‘epic’ story above, the team thinks about the various steps the user might take from the beginning to the end of the epic. A great way to get into these details while maintaining a specific and testable perspective is storyboard- a sequence of simple sketches that describe the experience you want the user to have. Here’s an example storyboard from HinH for the epic above:
‘As Ted the HVAC technician, I want to know the pricing and availability of a part that needs replacing so I can decide my next steps.’
The storyboard starts with Ted finding that he needs a replacement part. The next three panels describe three different ways the team observed tech’s identifying parts: a) by part number 2) by make/model of the unit and 3) by sending a picture to the office for help. In the last two panels, the Ted finds the part’s pricing and availability and uses this an input into his discussion with the customer about their next steps.
The team then structures a series of ‘child’ user stories that further detail the user experience within the epic:
| USER STORY | NOTES |
| I know the part number and I want to find it on the system so I can find out its price and availability. | Make sure it’s possible to search by part number. Make sure descriptive info. appears as the search narrows (photo?) to help avoid error. |
| I don’t know the part number and I want to try to identify it online so I can find out its price and availability. | Make sure it’s possible to search by make/model of units Make sure it’s possible to search by type |
| I don’t know the part number and I can’t determine it and I want help so I can find out its price and availability. | Make sure an estimate of the turnaround time for an expert to review is available |
| I want to see the pricing and availability of the part so I decide on next steps and get agreement from the customer | Make sure it’s possible to dispatch a request by email to the customer in case they order their own parts and/or carry their own inventory of spares. NOTE: How would the customer respond so we can help structure the next steps as we would otherwise? |
A Disciplined Process
Frangelico is happy with the process his team has run so far. Rather than giving into the temptation of just running off and building (or buying) software, they invested in just enough design research to find a problem/need/job that a) existed for the user and b) where the current alternatives weren’t strong. Following that, they made sure they really understand the details of how a solution might work with a concierge MVP.
Designers often reference the idea of ‘finding the right problem’ before diving in and building a solution. The idea is to ‘diverge’ and consider a number of options before ‘converging’ on where you want to focus, and to use the right tools at the right time to do this. Donald Norman formalized this idea with the ‘double diamond’ pattern you see below, along with notes on how the HinH team applied this pattern:
Testing and Iterating for the Win
Frangelico knows that it’s easy to get lost in the detail once you start development. He wants a clear set of metrics to help focus the team and to use in instrumenting observation into what they build so they can keep up the practice of making evidence-based designs and decisions.
Testing the Problem Scenario/JTBD
For the team’s focal PS/JTBD of ‘Getting replacement parts to a job site’, the core engagement metric is the number of parts ordered. Their hypothetical value proposition is:
If we automate parts lookup and ordering online, then the tech’s will use it and it will improve outcomes.
As Frangelico works to refine the team’s charter, he wants to make sure their operating on an analytically solid foundation where they can make evidence-based decisions about what’s working ,what isn’t, and where they should focus.
- How might they focus their observations on whatever solution they might deliver across these three phases of the user journey: 1) Onboarding 2) Engagement and 3) Outcomes. For each of those three phases, consider the following:
What does this mean?
What is the interval?
How might we test this?
What are the metrics?
What’s tricky? What do we need to watch?Note: Exhibit A below has a summary of this and the video at the end of this section describes an example. - The team has picked a specific slice of their value proposition to design and code in their first iteration. The slice is described by this epic user story: ‘‘As Ted the HVAC technician, I want to know the pricing and availability of a part that needs replacing so I can decide my next steps.’.
How does that epic fit into the larger picture of delivering their target value proposition? For the individual user stories within the epic, what observations would you want to instrument into the application the team will be building? For example, in the case of code delivering on this child user story, what metrics would you want to collect: ‘I know the part number and I want to find it on the system so I can find out its price and availability.’?
Note: Exhibit A only pertains to testing the PS/JTBD. For the user stories, you can just make notes on what observations you’d want to make regarding interaction with the individual stories.
This video shows an example of what answer might look like for item #1, testing the JTBD/PS:
Exhibit A- A Framework for Testing Problem Scenarios/JTBD
The following table summarizes the questions the teams should answer as they test their value proposition relative to a target PS/JTBD.
| Onboarding | Engagement | Outcomes |
| What does this mean?
What is the interval? How might we test this? What are the metrics? What’s tricky? What do we need to watch? |
What does this mean?
What is the interval? How might we test this? What are the metrics? What’s tricky? What do we need to watch? |
What does this mean?
What is the interval? How might we test this? What are the metrics? What’s tricky? What do we need to watch? |
Exhibit B- Interview Guide (User Discovery)
This is an interview guide the team developed to go out into the field and interview technicians with the goal of understanding their work and, in particular, where the was the most tensions between how things work now vs. how they’d work in an ideal environment. A few things that are notable about the guide:
- It’s just a guide. The team may or may not use all these questions. The goal is exploration, not for example, to compile data they can specifically compare on tightly defined features or attributes.
- It moves from general to specific questions. This is to avoid leading/prompting the subject for a particular answer. For example, the evidentiary value of hearing that getting replacement parts is hard as a response to a question like ‘What are the top 5 hardest things about completing a job?’ is much higher than just asking ‘Is it hard to get replacement parts?’ and hearing a ‘yes’.
- The output of this is a series of interview transcripts which the team will then synthesize into working personas and problem scenarios/jobs to be done.
PERSONA HYPOTHESIS
| Question Format | Example Questions (HVAC in a Hurry) |
| Tell me about [yourself in the role of the persona]? | Tell me about being a tech at HVAC in a Hurry? How did you choose that line of work? Why? What do you most, least like about the job? What are the hardest, easiest parts of the job? |
| Tell me about [your area of interest]? | What’s the general process for finding out about a new service or repair job and completing it? Can you tell me about your last job? Who else was involved in getting that job done? How did you all interact? |
| Tell me your thoughts about [area]? | How are things done now vs. how would they work in a perfect world? What have you seen work well vs. not well on specific jobs? What’s the best vs. worst job you’ve done recently and why? |
| What do you see in [area]? | Where do you learn what’s new? What others do? Who do you think is doing it right? |
| How do you feel about [area]? | What motivates you? What parts of it are most rewarding? Why? Tell me about the last time? What parts, if any, are demoralizing? What would it be like in your perfect world? |
| What do you do in [area]? | How many hours did you work last week? Is that typical? How many customer jobs was that? What are your favorite tools for getting jobs done, information-wise? |
PROBLEM HYPOTHESIS
| Question Format | Example Questions (HVAC in a Hurry) |
| What are the top [5] hardest things about [area of interest]? | What are the top 5 most difficult things about doing good work? Why? |
| How do you currently [operate in area of interest- if you don’t have that yet]? OR Here’s what I got on [x]- is that right? | How do you currently prepare for a new job/dispatch? Who does what? How does that work? |
| What’s [difficult, annoying] about [area of interest]? | What’s difficult about making sure you have what you need to finish a job? How do you validate they have the right skill set? How are the actual outcomes? Examples? |
| What are the top 5 things you want to do better this year in [general area of interest]? | What are the top 5 things you want to do better this year? |
| Why is/isn’t [your specific area of interest on that list]? | Why isn’t [x] on your list? |
Exhibit C: Persona for Trent the Technician
Screening Question: How many HVAC repairs did you complete last week? [threshold: >3]
Trent’s been an HVAC technician for 7 years; 4 years of that at HinH. After a couple of years off, Trent enrolled in a 2 year program on the advice of a close friend who was doing the same. Getting his first job wasn’t as easy as everyone said- and the money was terrible. But now he’s a senior technician at HinH and the money’s pretty good.
He’s always had a knack for fixing things, probably owing to his concentration, curiosity, and tenacity. He’s never been much for debating or arguing. He likes figuring things out and even more than that he loves coming through for people. More than his billings on a job, that’s what really makes it or breaks it for him- whether the customer is happy and better off or whether they’re confused and frustrated.
His day starts as late as it can- he prefers to sleep in but usually has to get to the office early. He prepares for his jobs and then hits the road as quickly as he can to avoid traffic. If he’s lucky, he has the parts he needs for the job and doesn’t have to deal with the logistics of getting them. If he’s not, there’s a lot of time on the phone and sometimes back and forth, and not all of it is billable.
He knows dispatch does the best they can, but sometimes he feels like he’s zig-zagging all over town and spending most of his time in traffic, which doesn’t pay- as a senior technician he’s mainly paid on his billable hours. Sometimes that gets pretty frustrating.
He tends to use his own mobile phone at work for email, text, and looking for documents online. The company provides a tablet-based device, but it’s kind of hard to use and he just refers to it for dispatch and a few other things.
| Thinks | Trent thinks the dispatch process should be more systematic to avoid jobs that are far away or not consistent with his expertise. Also, he wonders if there isn’t a better way to stock and distribute parts- that’s a big problem. All this is important because a lot of his time is wasted and he’s paid hourly for jobs. |
| Sees | Trent sees that a lot of the company’s best talent either goes into business for themselves or goes to work at large clients. They often end up with better hours and better pay. Sometimes, though, they end up short on business or having to put in long hours to do their own marketing and admin stuff. |
| Feels | When Trent’s sitting in traffic for a job he could have gone to earlier that day in 1/3 the time he feels angry, he feels cheated and like the company doesn’t care. |
| Does | Trent works around 45 hours per week. He’s regularly on his personal iPhone looking up equipment manuals since it’s easier than the company-provided tools. |
