Josh Andrews (Google) on Killing Complexity

Voiceover: Welcome to the Interdisciplinarian, where product managers share their stories and insights from the field.

Alex: Thanks for joining me. I’m your host, Alex Cowan, and today you’re going to meet Josh Andrews, who is a product manager at Google. One of the products that Josh has worked on is Google Apps for Work, which is their suite of communications and office applications: online spreadsheets, documents, e-mail, stuff like that. I use Google Apps every day and I love it. It’s simple and it gets the job done. You’ll hear a lot about well-liked products that, well, they kept it simple, and it seems like, well, that’s simple to do, but, of course, anybody who’s really been in there doing product management knows that it’s not. We’re going to talk about how you achieve that simplicity, which is hard. It takes work, it takes focus, and it takes great teamwork. You’ll hear about that from Josh today. Josh, thanks for joining us.

Josh: Thanks, Alex. Glad to be here.

Alex: You’re in sunny Mountain View at Google.

Josh: Yes. Yes, I am. Yeah. The weather is fantastic here.

Alex: You’ve been a product manager at Google for how long?

Josh: Just a little less than four years.

Alex: Tell us about your work on Google Apps for Work. What are the specific parts of it that you’ve worked on?

Josh: Sure. I worked on Google Apps for Work for about two years of my time at Google. During that time, I focused primarily on growth-related projects for Google Apps for Work, specifically focusing on our online sign-up flow, so this is basically the process where somebody would come to a website to learn more about Google Apps for Work, and then decide to give it a try, get Google Apps for Work set up for their company, and then, ultimately, decide to pay or not pay for the service.

Alex: Your job was to improve on that experience, make it better for the user, and increase the rate at which people actually converted and signed up for the service.

Josh: That’s right. Yeah. Ultimately, we looked at how much revenue we were creating through the flow, how well we were doing at converting people that came in, and how they rated their experience, how simple it was to set up, and how delightful it was, how well they would recommend it to people.

Alex: I’m sure you had to manage through a lot of things that needed to happen for the user to get them signed up. As any practicing product manager knows, it’s really easy to just take all that complexity and dump it in the lap of the user, but I’ve signed up for Google Apps. Clearly, you didn’t do that, so tell us a little bit about how you kept things focused, and how you kept them simple and natural for the user.

Josh: Sure. One of the biggest challenges with what we were doing was around e-mail. It’s surprisingly complex to get e-mail set up and working for domain. Basically, that means if you want me@mycompany.com to work, then we have to do a lot of backend routing between Google and whichever company you purchased your domain from to make that work. We found that there were, around the world, many, many thousands of companies that managed domains, that their customers were coming to us looking for an e-mail solution, and we found each one of those required, often times, something a little bit different, so we wanted to make it as easy as possible for the user, but what seemed to be very simple when they come through and say, “Yeah. Yes, I would like my mail, and this is my domain,” in the background, we actually had to look at where are you coming from and provide the easiest solution.

We ended up finding various solutions. Sometimes, we would set up a business-development partnership so that we could interface with other companies in the background. Other times, we would provide very specific instructions so that people would know exactly what to do.

Alex: When you say a business-development partnership, you mean some kind of automation or integration between yourself and the domain provider?

Josh: Exactly. That’s right. Yeah. We found the highest-volume domain providers that we were dealing with, and we would go and we would talk to them and try to set up a special relationship so that, essentially, all of the complexity would be removed from the user, and they would just say, “I would like e-mail,” and then we would say, “Okay. Here you go.”

Alex: Tell us about how you generated for yourself all those different options, and how you might engage with some of these providers. Were those sort of readily obvious, or did it take some work to think through those with your team?

Josh: It’s not always obvious, because I think I’m not an expert at e-mail routing for domains and that entire landscape, so generally, it was a collaborative team effort. Basically, we had many different people from our support organ, from our engineering organ, from our business-development organ that all knew that we were going after this kind of a solution, and then everyone would work together to generate different ideas, and then we would all work together to see which of those were the best and which were the most feasible to do, but it’s something that you have to do as a team, because it’s very difficult for one person to have enough knowledge to figure all of that out.

Alex: Was there anything in particular you did as a product manager to generate those perspectives? You just get everybody together, or present the problem on e-mail or forum, or what worked best for you in terms of actually generating those options?

Josh: Usually for those, I like to start with what’s the overall vision, or what’s the reason why we’re trying to do something. The more real you can make the problem, and the more real you can make what it’s going to be like if you solve that problem, the more engaged people are, because it’s very easy to go ask somebody to do something, and then just to not do a good job of it, or just not do it at all because they don’t think it’s very important, so I always try to start there. Then, besides that, it varies somewhat. I would say sometimes, when I would work with people, it would be as simple as setting up a meeting, walking somebody through something, and then dealing with things over e-mail. Other times, you would get like-minded people together in a meeting and try to do a group brainstorm. Sometimes, I would meet with a manager who has many people that work for him or her, I guess, in this case, it was a him, and try to enlist that person to then get more of their team involved in identifying potential solutions.

Alex: Let’s go into the details of an individual implementation. Can you talk a little bit about how you would take one of these options for a specific provider and move from whatever input you used, user story or whatever, to all the different details, to implementation, to testing to make sure that this was really generating good outcomes for the user. Can you talk a little bit about that process?

Josh: Sure. If you imagine one of the ones where we had to set up a business-development partnership, in that case, once we had formulated the idea, and we had a rough sense of what we want, we would then go work with someone on our business-development team who would then contact out to the third-party companies to actually try to engage and see if they’re interested in setting up a deal. Then once we saw that interest, we would put them together with our tech team, and we would walk them through what the interface would like and how it might work, and then once we had both of those things scoped out, then the external company and then us internally would both work to build that kind of a solution out.

Once we had that basic integration working, then we would start trying to bring it back into our core set-up flow that we had, and basically, what we would have to do is look at where in the flow things would get different for someone who fits into the segment that would apply to that partnership, and then create an invisible separate path that that user would take. At that point, we would generally start off with listing off requirements, looking at documents. I would then work with UX designers to imagine how something like that might work, and then we’d do that in collaboration with the engineering team to figure out how to deal with-

Alex: How did you go through the process with the UX designers? Did you guys sketch? Did you prototype and test with users?

Josh: Yeah!

Alex: Did you already have patterns that you built up and you could just use those?

Josh: I would say some of all, but for the most complex things, generally … In this case, what we did was … We always start with some core-user stories, which are around what’s the problem and what’s somebody trying to do. In this case, it would be I have a domain with this company, and I would like to get Google Apps for Work working on this e-mail address. Then, at that point, we would sketch out some basic ideas that would solve the problem, and generally, try to look at a few different extremes of an idea. I think in this particular case, we tried to come up with four different ways that we might try to solve the problem, and then we created some very lightweight prototypes, basically, things that someone who knows basic website programming could mock something up in a couple of days.

Then we got people that were within our target demographic that would look at this over a hangout video, so basically where we could share our screen with them, and they could review it and give us feedback. We would let them play around with our prototype, and we’d get feedback. We did that with probably five or six users for each of the four different variations. We used that to get the feedback.

Alex: Tell us about the variations. Why do you think it was important to generate four variations, or five, whatever it was?

Josh: Well, I think the easiest trap to get into is thinking that you know too much. If you don’t do a number of variations, then you risk biasing the feedback that you get, so I think when we think about doing variations, there’s often one that we think is our favorite, but then we try to identify different choices that we’ve made, and try to push that choice in a different direction to create a different variation, because then, often times in these, you just don’t know what you’re going to learn from people that are actually living the experience. It’s often times very difficult to empathize with everybody and really get into their heads.

Alex: Once you’ve sketched these out and tried them out, how did you move forward and decide on what you were going to do?

Josh: Based on the user feedback, we get a sense of what we think is working the best, and we will then narrow that down to one of the variations, and then get into a higher level of detail, a higher level of fidelity, for those, and then we actually bring in the more heavy engineering, but we always try to keep things as light as possible until we get better market validation. In this case, that means actually putting something in front of a significant number of users that are actually trying to sign up for Google Apps, because when you’re doing user studies, these are not people that are actually in this situation, so you don’t want to take that with too much faith, so we would generally try to build a half version, or a light version, of what we ultimately wanted to build, and try to get that out in front of real users.

We would then run a split test, or an AV test, where we look at segmenting the people that are coming into our flow, and then we’d give 20% of them the new experience, and then another 20% of them the old experience, and then we compare who’s more successful at setting up e-mail for their domain, and we see did we do a good job or did we not do a good job.

Alex: When you say a light version that you’re able to iterate on quicker, what things are you able to defer until you get tighter validation that would make it lighter? How does it differ as you lock it in?

Josh: I would say that’s usually a conversation you want to have with your engineering team, because there’s no one answer for that. I think in this specific case, we did not handle all of the error cases very well, so there was many potentials that somebody going through this would have a bad time, potentially. Then we also didn’t build it for large scale. There’s a lot of engineering that goes into being able to deal with something at a very, very high volume, but since we’re running a small-percentage test, we don’t have to do all of that work. Both of those things greatly reduce the amount of engineering work that goes into something.

Alex: Now, something like this, obviously, you don’t do it by yourself. What kind of talent is on a typical team that you’re working with to do something like this? What are their roles?

Josh: It varies a lot from project to project, but usually, when we were dealing with something like a set-up flow for Google Apps for Work, I worked with other product managers that would work on kind of adjacent systems. I worked with UX designers and engineers, as I talked about. We also had a marketing team, which managed the entry-point website and a lot of the e-mail content that users would get sent during their trial period. We also had an operations team that would deal with the phone support for people that had issues. There’s more operations people that deal with backend stuff. There’s salespeople that can get involved sometimes for larger customers that are using the flow.

Alex: Got it. With a team like that where you’ve got a lot of interdisciplinary expertise, as a product manager, how do you keep the work focused on what you think’s going to be valuable to the user while still cultivating the creativity and the initiative with the team that you’re working with?

Josh: This is a constant challenge, and something that I know I have not always done well. I think most of the time, in these cases, what I try to do is keep things as simple and focused as possible, which means that we know what users and what use cases are we going after, so in a case like this, this means that, for instance, we might not be designing an online sign-up flow for a company that has 10,000 employees. You want to make sure that everybody knows what you’re going after and why, and then what the goals are, and then make sure that you’re meeting with people, you’re talking with people, you’re sending them updates, so that they see that there is momentum, and there is a goal, and things are happening to make that happen, and then I think that helps other people see that they want to be part of the thing that’s happening rather than do something else.

Alex: Do you have any other advice for listeners who want to get better at keeping things focused and delivering nice, simple, highly usable experiences for the user?

Josh: The best thing is just to make sure that you’re starting with your users or your customers first, and really ruthlessly prioritizing what you go after and making sure that you do a small number of things well rather than a large number of things poorly. I think that’s usually the best thing that I try to do.

Alex: Before we close, I’d love to talk a little bit about your career story. Before business school, you were the development manager at Banker’s Toolbox, which is a software company. What was that like, and what did you learn there that’s been helpful to you in your roles subsequent to that?

Josh: I started off as a software engineer at Banker’s Toolbox. Then I grew into a development manager, and was a product manager for a while there as well. It was a relatively small company, so I found that there was lots of needs. At a small company, people tend to naturally wear many hats, so I think one of the things that I found there was that it’s important to identify problems and try to go after and solve those problems regardless of whether or not they’re your job or in your domain.

I think that’s something that’s been very helpful for my career now as a product manager at Google, because we’re kind of the garbage men of products. We’re there to set an initial direction, but when it comes down to things that I actually do, as an individual contributor, it’s almost always dealing with the leftover stuff that other people don’t want to do, or can’t do, or that we can’t find somebody to do, so I think it’s served me well to work at a small company, and to really have a passion for just doing anything and everything [inaudible 00:18:12].

Alex: Then you went to business school at UVA Darden. Imagine you are the Josh of six years ago, I think it would be. You’re just starting business school. What advice do you have for that version of yourself? Where would you tell that Josh to focus and learn things while in school?

Josh: I think that the two things that have been most helpful for me in my work life after getting an MBA degree have been things that I learned around leadership, and things that I learned around communication, and I think I would make sure to take more of those in business school, and spend more time talking with professors, and thinking through a lot of different scenarios there. I think beyond those, I had a great time in the negotiations class at Darden, and would recommend that. I find I’m constantly doing negotiations with people. They may not be things that you would think are a negotiation, but I think one of the things I learned in that class is that lots of life is a negotiation, and lots of work is negotiation.

I think, beyond that, one thing that I think I would have liked to spend more time on Darden that I really didn’t was learning how to develop and cultivate a network of people. I think I never really thought of that, really, before I got to Darden, and was a little bit skeptical of it while I was there, but I found more and more in my work since then that that’s one of the critical advantages that I can have when I do my work, is having a strong network of people that know me, and like me, and that I have helped, and that I can call on to help me at the right time.

Alex: This has been great, Josh. Thank you so much for joining us.

Josh: Thank you, Alex.

Alex: A couple of the things that I thought were particularly interesting about what we heard from Josh is the need to absolutely focus on what’s really important to your users, and try to do a few things really well rather than trying to do a lot of things, and probably overloading yourself and your team with too many things to do. You’ve also heard a lot about how they work from this core idea out to a working implementation, and how they challenge themselves to test multiple versions of this product so that they don’t bias themselves, and they don’t bias their testing, to one particular idea that they start to like internally as a team.

If you’d like to learn more about building great products, I can’t help but mention our agile development specialization on Coursera. You can check it out at bit.ly/hiagile. Thanks for joining us.

Voiceover: The Interdisciplinarian is a production of Darden Media, in cooperation with the Batten Institute at UVA’s Darden School of Business.