Christopher Monnier, Design Researcher - Airbnb

Now is the moment to ask you a question. If you ум Clear me Done I love the sun. Like everyone. I love the sun. Like everyone. Like a sleepy one. Each on his own. Like a sleepy one. Like a sleepy one. Like a sleepy one. Like a sleepy one. While I was playing fair, baby. You played a cheating game. While I was playing fair, baby. You played a cheating game. Thank you. Thank you. Good morning, everybody. Yay. Please have a seat. We still have some really good rows down here. Thank you. Thank you. Thank you. Good morning. Well, good morning, everybody, and thank you for coming. Welcome to Design Matters. I'm just going to give the last ones a couple of seconds to settle in, but it seems like we're almost there. All right. Good morning, everybody. And welcome to Design Matters. My name is Ingrid. I'm one of the arrangers, the organizers of this conference, and I'm super, super happy to be here. And I'm very excited to meet all of you, all the speakers, all the audience, and spend two days with you here at Talgul. So why are we here? Well, of course, we think that design matters. It really, really, really matters a lot. It matters. In everything we surround ourselves with in our daily lives. It matters in the little things that provide us with function. And not the least, it matters in the devices and screens we look at a million times a day. We wouldn't look at them if they weren't well designed, would we? So how does great design happen? Well, often it evolves a little bit like this, like the refrigerator example here. Step by step. Each new version a little bit better than the previous one. Better aesthetics, better technology, perhaps a better price. One improvement at the time. And for each improvement, a bit more value is created for the user, the customer. The users start using the products in slightly new ways. They get a little bit more empowered. They feel that they get a little bit more comfort. Suddenly there's a cup holder. But fundamentally, the product is the same. And the market evolves gradually, step by step, without really changing radically. And this step by step innovation or design is usually referred to as sustaining innovation. You probably all know that. But then, sometimes products change our ways radically. They disrupt our usual ways of thinking. Of acting. Of socializing. Of behaving. And that's called disruptive innovation. Pow. As you probably all know, because otherwise you probably wouldn't be here, but I'll tell you anyways, is that disruptive innovation, it changes the market that it gets to. Or it creates an entirely new market. It changes industries. It changes sectors. It puts people out of work. It destroys companies. And it creates new ones. We often see that entirely new players emerge on the field that was long occupied by old companies and corporations. To give you kind of a fun, old example, I'd like to talk about this picture. It's taken around year 900 in a town called Hudson, two hours north of New York City. It's by the river, the Hudson River, that also flows through New York City. And in the year 1900, there was a large and thriving ice industry in Hudson. Workers in the wintertime would go to the river and ice fields, and they would pick up blocks of ice. And they would store it in ice houses, just like you see there, along the river. So in the spring and in the summer, they would ship down those ice blocks to New York City for the ladies to use in their, I don't know, their drinks or their food closets. And they would bring it on boats. So it would just kind of get ice from upstream, down to New York City, all through the summer. But then, of course, as things had to be, in the 1920s, the refrigerators started to come. And suddenly, they limited the need for ice from Hudson. So refrigerators disrupted and killed the ice industry in Hudson. That was in the old days. This is also in a bit of the old days. But then, when, in 1900, the ice industry was destroyed, in 1992, the World Wide Web came. That disrupted, I'd say, practically every market for sharing and reaching information. It also disrupted our ways of shopping, of dating. I mean, the list is endless. And then in 2007, when the first user-friendly smartphone came, I'd say, that made the internet mobile. And these things have changed so many ways, our ways of behaving and acting. It's changed so many markets. And currently, I have the feeling that, in the digital world, disruption happens more than ever. So what about disruption and design? Is design the key factor in disruption? Of course, price and technology are essential. But so is design. Because if design wasn't there, would the disruption really happen? I'm not really sure. Because we all know that users want to change their behavior if they get a user-friendly, attractive, pleasurable product. And that's all due to design. A great example is this one, the Nest thermostat. I've never used it, but I've heard so much about it. It's supposed to make people feel comfortable. in very complex ways that they never have before. People that have never had a behavior panel related to thermostats suddenly start doing things they hadn't done before. That disrupted that market, or it will maybe. Because it's not that outspread in Denmark yet. The Fitbit wristband. That created an entirely new market. It's not that people weren't tracking their steps and the activity before. It was very, very limited. Suddenly it became a mass market. So that disrupted that whole industry in just creating something new that wasn't there before. People would track their activity, track their steps, and that was just only for the very few before that. And who knows if we in the coming years or even months would see industries getting disrupted by this little fella, the Apple Watch. It just might. This conference brings together some smart and talented people that all contribute to changing the way we act and behave and think by creating great product. Comment for them is that they do something that actually disrupts our ways in some way or the other. And they do that in the way of designing something so well done that we actually would want to use it. And without that really great design that they're creating, I'm not really sure that disruption would happen as it did or as it is. We call them game changers. And you know, the industries that are being changed and being disrupted as we speak, or the last couple of years, they're probably also well known to you, and that's probably also why you're here today. But let me just bring up a few examples anyways. We don't really stay in hotels anymore, at least not when we go to New York, because we go with Airbnb. Right, Chris? And of course, we don't do any international phone calls anymore, because you're stupid if you don't do Skype instead. In Denmark, we don't call a taxi anymore. Actually, we still go with taxis because Uber is not here. But we don't call a taxi anymore. We don't sit in line and all these things, because we just use a little nice app called Driver. And do you remember standing in a line in a bank to transfer money to people? It might be a bit a while ago, but everybody in Denmark uses mobile pay now. And these guys, remember them? The marketing team. I'm sorry about the photo. I think they look like a marketing team. Totally. We don't need them. because it's so easy to use MailChimp for your next mail campaign that we can just do without them. And last but not least, we don't have to guess anymore when to and when not to, because women just use Clue. And that's just a few of the game changers we're going to meet for the next two days. They're going to talk about their products and how they're designed and make the choices they create, and create great experiences. They'll talk about how they change people's behavior, one design decision at the time. And I'm really looking forward to hearing everybody and listening to all the great speaks and to make sure that that goes exactly as planned. I'd like to introduce our conference chair for this conference. Her name is Helle Martens. She's a UX designer. She's worked in the industry in Denmark for many years. She was a part of Nokia when Nokia was ruling the mobile phone industry. And she's been working her ass off organizing this event. So please give her a very warm welcome. Very warm welcome, Helle Martens. Is this working? Yes, this is yours. That's Chris'. Sorry. Thank you so much, everybody. It's not just me. It's still here. Ingrid and Michael had a big part of this, too, and we've had a blast doing this. I'm so amazed at all the people here. Even, I mean, this list of speakers is amazing. We've got just incredible speakers. Incredible that everybody here came here today. We really appreciate everything. Thank you. And thank you to our sponsors. We have One.com, who not only flew in Skype, Ross Smith from Skype. They've also given you all a year free hosting, if you can use that. It's in your notebook. There's a green notebook with a little sticker on it. Thank you to Pentia and Operate and Think Digital for hosting workshops. We're doing those tomorrow. And thanks to 23. This entire session, the whole conference is being recorded. And since we were already recording, we have a surprise for you. We're actually streaming live. Hi, Mom. So share this link with your colleagues if you would like. Now I'm two slides behind. And this has also just been sent to your emails if you'd like. Thank you to Exure and Balsamic and Lomax, who've all provided merchandise and sponsorships. And thank you to Proto.io, who have generously given each of us a 60-day trial. And they've actually also given us two free one-year subscriptions. And we'll be competing about those. We're having a little competition. So if you hashtag your photos on Instagram, we'll have two winners, one today and one tomorrow, for the best Instagram picture we have a jury. So, thank you. If you're interested in a Proto.io one-year subscription, please go ahead and tag your photos. Practical information. This is our hashtag. There's a wardrobe where he came in. I'm sure you've all seen it. It's only us on the premises, so there's nothing to worry about, even though it's not guarded. Toilets are either up here on your right or out by the wardrobe. The weather is fantastic today. Thank goodness. So we're having lunch outside in this area. But this area is only going to be opened up. And if we're going outside otherwise, we need to use this area out here. Also, if you've signed up for our speaker's dinner tonight, you'll notice a little clip on your tag. If you're missing that, please talk to Michael, the young fellow over here, and he'll help you. And if you haven't signed up yet, we still have a few tickets left. And that's also Michael you need to talk to. Now, I'd also like to introduce you to our other house. We've got Michael, Ingrid, myself. If you need help, please talk to either Vicky. She's over here behind you. Hi, Vicky. And Mia. And we have another Mia who's coming later. And we've also got Katrina. I'm sure you noticed her at the entry. She had long blonde hair. Beautiful lady. I don't know where she is. But anyway, any of these people will help you with anything you need. We've also set up a lounge area in case any of you need to break out and work. places to sit either out by the wardrobe, this little area by the bar, or in the back room. You can also break out if you need to. So now I'd like you all to find a person in your vicinity that you don't know and just introduce yourself and spend two minutes, five words or so, explaining to them why you're here. Please go ahead. I need to do an introduction of you first, though. Okay. Two seconds. Okay. Was that about two minutes? It sounded like a lot more than five words. I'm going to need to ask everybody. Your minutes are up. Okay. Excellent. Thank you. I'm going to ask everybody to please remember what you just said and then fill that out in our survey after the conference, please. We'll be sending the link out tomorrow. And we'll also be sending a link to all of the recordings that we have today. So the program today, we have today, we have Airbnb, Plan Day, and Volvo before lunch. And we're having a wonderful lunch outside. After lunch, we're running a fast-paced one-hour session with three new startups, followed by mobile pay, and then afternoon coffee and cake. Our afternoon is going to be really intense. We've got Driver, BBC 23, and Spotify. And after each of the presentations, there'll be room for a short Q&A. And now I'd like to talk about our first presenter, Chris, Chris Monier of Airbnb. Chris and I have a lot in common. We're both mechanical engineers who ventured into UX. We have both lived in Denmark and in California, and we both have a son named Oliver. Awesome. Chris calls himself user experience generalist, and he has extensive experience in all areas of many aspects of UX. He's been with Airbnb as a UX researcher for about a year. Before that, he actually worked for Novo Nordisk out here in Copenhagen, Denmark, doing design research and facilitating innovation. He loves turning complex challenges into enjoyable user experiences. All the way from San Francisco, please give a warm welcome to Chris Monier. Thank you. Thank you. Does this work? Hello. Okay. Is that appropriately loud or quiet? It's good volume? Okay. Okay. No. That's it. Nope. , no. All right. Cool. Yeah, thanks. Thanks for having me. So I work at Airbnb. Specifically, I work on the growth team, which is a subpart of Airbnb, which I'll get into as I go through this. Like I said, I'm a user researcher, so I'm not a designer. We have awesome designers at Airbnb. There's like 50 of them. We have a team of like 12 user researchers. I'm one of them. So this talk covers, it's sort of coming at it from the research angle and how research influenced the design that was done by the awesome designers. So like everything that you see in here is a total team effort. I'm just sort of representing it. So just an overview of what I'm going to talk about. Hopefully you can read that. This is one of like the Airbnb brand colors, which is why I used it. So despite the bad contrast. Looks better on my screen. So I'll just go through a quick background of what I mean by the growth team at Airbnb. The process that we use, like the product development process. Just a quick like behavior change model, just like, just to sort of provide some context for a couple of case studies that I walked through of like actual designs that just got launched, like I don't know, within the past couple of months. Definitely feel free to ask questions throughout. Like don't worry about interrupting me. There'll probably be time at the end for questions too. But if something doesn't make sense or you can't read it because of my poor contrast, let me know. All right, so background on the growth team. So what is growth? And isn't it all growth? Like why is there a growth team? Like when I first started Airbnb, I heard there was a growth team and I didn't really know like why. Like I was like, but aren't we all trying to grow the business? Like especially a startup, like what's the deal? The difference is that most teams at Airbnb are focused on bookings or nights booked. That's the main metric. Like that's how we judge as a company. That's how we judge whether we're on the right track. But the growth team's key metric is the number of registered users. So like in a nutshell, we're just focused on signups. We want to get more people into the top of the funnel. Like funnel metaphor. More people into the top and then it's other teams' responsibility to sort of like convert them into people that actually like book a room. So being on the growth team and having our metric be signups is really interesting because it frees us up to work on parts of the experience that otherwise would fall through the cracks. So some of those things are like referrals. So it's like if you share, if you refer someone, you both get $25. Engagement emails. It's like if you are on a page and you almost book but you don't and you get an email suggesting that you should. Viral sharing. So that's like sharing through Facebook, Facebook Messenger, WhatsApp, WeChat, whatever is relevant in your country. Search engine optimization. So that's just making sure people can find us on Google. And then also some like bigger picture stuff like group travel planning, which I won't talk about today. Maybe we'll talk about that next year if I come back. Who knows? Anyway. So like I don't know if you've seen. The growth, growth teams in general have this reputation like whether it sort of started at Facebook. This idea of like a growth team. And there's this reputation that I have of being very like tactile and sort of not tactical and not focused on like the holistic user experience. And it's something we really try to be different at at Airbnb. This is a quote by Brian Chesky, one of the co-founders and the CEO, which is love creates growth, not the other way around. And so in other words, we try to be different. We're not trying to like just get some short-term gain at the expense of like long-term love. Everything we do on the growth team is sort of in the larger context of trying to make a better product for people. Something that people love for the long term. So this is process up here. I'm just going to walk through the process quickly. All right. So this is like a typical like software development organization. It's a general product development organization. If you can't read it, it says data science, which is like a team of like analytics people. Experience research, user research, that's me. We have content strategy people, people that write the words. Experience design, that's like a typical UX designer. Production design, they make it pixel perfect. And then engineering. So we have this broad team. And usually those people like report into like a product manager who's sort of steering the ship. And really what we find is like just the culture of the organization is that like everybody talks to everybody. And it's one of the essential, one of the reasons why a lot of the things that I talk about in this are able to happen in like a day or two. Like we just are so cross-linked that whenever an interesting idea comes up we're able to really act on it quickly. Okay. So just a quick model of behavior change. Is anybody familiar with this BGFog model? Has anybody seen it? So it's just a pretty simple model. simple model, but I think it's really powerful. So on this axis, there's motivation, like wanting to do something. And over here, there's ability, like how easily it can be done. And the idea is that if you're on the other side of this curve, if you're over here, then if you're triggered and you have enough motivation and enough ability, then you'll do whatever the trigger is sort of asking you to do. So you can sort of think of it in two ways. On one hand, if you want someone to do something, you have motivation and ability. You can either, say like here, you can either increase the ability, so just make it easier in the product. It's like a usability change or just making something more discoverable. Otherwise, over here, you can increase motivation, so you can make people want to do it more or give them an incentive to do it. Or you can, of course, do both. So the first thing I'm going to talk about is just a quick case study that talks about like a simple change that increased ability, but had a pretty dramatic effect on stuff. So in this project, the goal was to increase the number of times that a listing, that's like a place you can stay, a listing gets shared. So we want people to share listings with their friends, maybe help them plan a trip, or maybe post it to Pinterest or Twitter just to get like general attention to Airbnb. And here's what we did. So this is on Airbnb, like you click on a listing, and then there's like this photo, these big pictures, you can click through the photos. And then this is like the rest of the page. Then there's, oh my gosh, there's like, this is like a bunch of text, which describes the listing. If you can't read it, it's like, it has like things like guest access and like house rules. And sometimes people write like huge paragraphs of like what you can do on a listing. But we wanted to see how people were like using this page. So we did a variety of reasons. But one of the things was like, just to understand like where people's attention was. So I did some user research. This is Skype. At Airbnb, it's cool because we have like, we have this database of like millions of users. So we can easily reach out to people and ask them if they're planning a trip and then do like studies with them. Sometimes we do it in person. Sometimes we do it remotely. In this case, it's remotely. This guy is in the UK. I'm in San Francisco. He's just sharing his screen. And basically the task I gave him, which I wanted to see how he did, was like, look for a listing, like, where are you going on vacation? It was somewhere in Scotland. And then show me like listings for that. Like, show me, show me how you would go about finding interesting listings. I just wanted to see where he was focusing attention, focusing his attention. And so I did that with like a variety of users and then came up with this overall sort of understanding of how people use that page. We call it P3. It's also called, it's the listing profile page. And this is sort of the general pattern that emerged from that research. So the first thing, everybody, everybody looks at the photos. Like it's just the first thing, of course, these big, nice juicy pictures. And then after that, people tend to skip past all that text down to the reviews. They want to know if it's like actually a good place. It looks like it's good. What is it? What do the other people say about it? And then they go over to the price. These are sort of like the three, it's a photos, reviews, price. We found that that order, of course, not always, but generally that's what people are interested in when they, when they hit that page. So yeah, everybody looks at the photos. So we thought like, that's, that's where people's eyeballs are. Like if we want people to share, let's put sharing there. One thing I, did I get to mention? So like, this is, this is that page. We have sharing options on this page. It's like, you can send an email, you can send it to Facebook, you can tweet it, you can embed it, you can do whatever you want. But that's on that page. And then we have, we have a lot of people that are like, oh, we want to share this. So we're like, okay, we just want people to share. So let's put it in front of their eyeballs. So we just made this little change where we put like before this didn't exist. And then we put like some, some simple sharing options up here. Same icons, email, Facebook, whatever. Um, uh, that's just a zoomed in picture of it. And the results were pretty impressive. So just putting those icons on that screen resulted in like a 20% increase in the number of people that share the listings. Uh, so it feels like a pretty basic example of how by making it just a little bit easier by increasing ability just a little bit, we like changed, uh, behavior quite dramatically. Um, 20% is like a pretty good jump, uh, for Airbnb. Um, any questions about that? It was pretty straightforward, I guess. Um, all right. So now this is like a juicier one, a more interesting one. I think this is, this is about increasing both motivation and ability, um, to do something that's even harder than just getting people to share. Um, so yeah. So like one of the things I mentioned was viral sharing. Like we're trying to get people to share, uh, both content on Airbnb, but also like the, the trips that they book. Uh, specifically we want to get, like if you're going on a group trip with like five people, we want you to, uh, to share your ideas. Um, and, and for us, like selfishly, it's, it's, it's an opportunity for like all those co-travelers to get an email on which there can be a call to action for you to sign up. So it's like, so like benefits everybody. Um, and specifically it's like a, it's an interesting tactic to get signups. So we want to increase the number of, travelers. This is a page that you hit after you make a reservation or after you make a request for a reservation. You can't really tell, but there's a bunch of form fields here. So it says like add the other guests and there's a bunch of like text boxes for you to type in the other guest names. And another one down here. This was like a pretty ugly page and we wanted to improve both like the aesthetics of it because we thought maybe that'll just make people engage with it more. But also like provide people with an incentive to actually do what we want them to do on this page. This page, as it is designed right now, it was just like arbitrary. It doesn't have really any intention behind it. So we did something pretty cool. So there's this tool called Ethneo, which is a really great way. Like if you have a website at all, even if the products you make are not like digital products, like Novo Nordisk, for example, you guys could use this if you put it on a website. Right. So this tool called Ethneo is really cool. It's just this little thing that like slides up and and it just says penny for your thoughts, blah, blah, blah, blah. You want a $500 gift card? People are like, yeah, sure. And they click this button. Maybe you've seen this on websites. Usually it just it's like a thing that pops up really annoyingly in the middle. And like you always dismiss it. But this sort of clever implementation just puts it a little out of people's way and actually gets like a fair amount of clicks. So this is like a good way to intercept people right in real time. When they hit this when they hit this page, which is super important for getting the right context of like the mindset that people are are the people have when they hit this page. So anyway, so yeah, we intercepted people with Ethneo to like 1500 people, which takes about a day on Airbnb because of the response rate of that. And then just give them a survey and ask them, what are your thoughts on adding the names and email addresses of the people traveling with you to your itinerary? It was a super open ended thing. We weren't trying to like, funnel it towards any particular solution. It's just an easy way to understand broadly like people's opinions of this task, which we wanted him to do. We wanted them to add names and email addresses. We just want to know how they felt about it. So like a bunch of responses that like open ended responses that I just read through and then coded into just specific categories. And the three top ones, the three that stood out the most were like, what are your thoughts on doing this? It's good because I get to share listing details. It's good because I can coordinate with people. It's good because I can stay, keep my other my friends in the loop. And it's good because it's easier for some people. At least it's easier to send the emails right there as opposed to waiting till you get the email and then having to remember blah, blah, blah. So these seem like really promising value propositions. We thought, oh, cool. We'll just, we'll like amplify those, maybe tell people about the benefits of this and then it'll work great. So we totally redesigned the page. We added, um, we changed the way that you enter email addresses. You can't really tell, but it's, it's similar to like, um, Google calendar where there's like just one row you can add and then every name you add just shows up down here. So instead of individual text boxes, we just had, uh, like one, one text box and then like whatever names you added showed up down there. We added a, like a big paragraph box for adding a message because, because we saw in the survey that people, uh, liked to coordinate with their friends and we thought, well, cool, we'll give them a message box and they can like type out like, Hey, this is a cool place we booked. Uh, excited to see you here. Um, and then there's another link down here for, um, letting other people know. So just to compare the two, uh, this is the old one and it's great detail. Um, so like this is a bunch of text boxes that are not very like beautiful. And this is like updated with like Airbnb brand colors. Uh, the blue things are just notes for me. They're not actually on the page, but like these nice, like this is like a brand color. Um, and we have like the right font and just looks generally nicer. So we were super excited when we launched this. We're like, this is gonna be great. Like people are going to love it. They're going to add, everyone's going to add their code travelers. Not at all. Totally wrong. Like you can't tell, but like this is zero. And we went from whatever it was like 20 some percent to like just dropped off a cliff. And we were like, what the heck? Like we totally, we had it right. Like we, we asked people what, why they did this. And we, we, we didn't give up. Of course we had to figure out what was going on. It was just amazing that we could like through that little change of the, of the user interface, just completely move the metrics in the wrong way. But it was still interesting that they moved so much. Um, so we did some like heuristic analysis just to see like what the deal was with the page. And we saw a couple of things that were like, that stood out as like, Oh, maybe this is why. Like, so maybe this slick, uh, Google calendar style implementation actually was like too slick. And maybe it's better to just have like text boxes for each name. Cause that's actually a pretty strong affordance, uh, for, for adding, for just adding stuff. If you see text boxes, I think the general expectation is like, I should fill those text boxes. And we took that away. And then people, I think, didn't know what to do. Um, the other big thing is, so we were like, we had this message box, which we felt so strongly about, but then it just was not working well. So we're like, is, is it, is the message box the problem? What's the deal? So we did more, um, research via ethneos. So we intercepted people on this page. Uh, and this time just did live interviews. This is like the coolest part of ethneos. It's like an ad for ethneos, but we love it. It's awesome. Um, like the cool part is like, you get that little like slider pop-up thing in the corner and then you can, you can put whatever you want in it. So we put like, uh, Hey, do you have time to talk right now for 15 minutes? You get a $25 Amazon gift card. Just give us your name and phone number. And a lot of people did and the cool, and then you can just call them immediately. Um, so, uh, like, so like two minutes after they see this on the website, you're talking to them, uh, which is like, it's like perfect for getting just the right context of like, cause we could, we could talk to them like a day or two later, but they wouldn't remember. I don't even remember this page. What do you mean? But, but this way we could intercept them right as they were going about this task, which is super cool. So anyway, I did that and I talked to people and I asked them like, we just showed them this prototype. What do you think about this message field? I realized that people were totally confused by that message field. They didn't know if it was a message that would be sent to the host because this is a picture of the host over there. Um, and the previous page, like when you make a reservation, there's a big message box to send a message to the host. So like it was just generally confusing. Um, so putting all that together, we thought we need to dramatically simplify this whole thing. Uh, this is like an Airbnb jargon thing, but what's the one thing for this page? Like we, we realized we were trying to, we were asking like three different things to people like, uh, send this to your co-travelers, add a message. Uh, here's, uh, your hosts information. It was just like, well, overwhelming. And people, I think we're getting, we're hitting the page, getting super confused and then just like bouncing, which is why we saw the huge negative spike. Um, so we thought, what's the, what's the one thing that, that this page is all about? We went back to this data and the, like the, the next thing we had, which we didn't try, was, uh, a lot of people mentioned this and this was just really interesting because we didn't ask, the question was, what are your thoughts on adding the names and email addresses of the people traveling with you to your itinerary? We didn't mention anything about the host, but like on their own, a lot of people, um, brought up that it's, well, it's good for the host to know. And we didn't even mention the host and the host actually doesn't get those names or didn't at the time. Um, but people brought it up on their own. So there was like, we realized that there was like this expectation that hosts would, well, it's only fair. Like we're staying in their home. Like they should know not only who I am, but also who my co-travelers are. That's, that's only fair. Um, so that seemed really interesting. We thought, well, let's try that. Let's go with that. Um, we dug into a little bit more. I did another survey on ethno. It's just like magic. You can just do whatever you want, whenever you want it. Um, so if you're ever on Airbnb, there's like a decent chance you're going to see this ethno tool. Cause I'm like always using it anyway. Uh, the survey, are you okay with telling your host the contact information that people are traveling with? Overwhelmingly positive, like 92% at least said it was okay to share the first name, which is all we really want, um, to give to the host. So we, we realized, okay, I think we're onto something like there's something about, uh, about this idea of sort of transparency and sharing information with their host that maybe people will respond to. Maybe that can increase motivation, um, to get them to change. So we sketched out an idea of how we can make this page all about informing the host who's coming. Like, let's make that the one thing of this page. That's like the job of this page. Is like people are going to hit this page and it's not necessarily like, Hey, um, be extra convenient and send this information to your co-travelers. Cause that's a nice thing to do. Instead we said like, you should send this information to your host cause they want to know. And like, and then there's like, like down here it says like also your co-travelers will copy this email. Um, so this is what the redesign looks like. Um, if anybody wants to see the details of this, just like find me later. I can show you like what, what the, what the details actually look like. You can't see it here. Um, anyway, so remember like this old, uh, this one is a text box, text box, text box. And this actually performed pretty well. Like, Oh, I forgot to mention. So, um, all of this is like, I showed that graph where like it just dove. That's because that was like in the experiment. So anybody that works the software probably knows this, but other people, um, we do these things called control. Um, you'll launch something in like half the number, like half the users in the world and the other half get the control. So when we launched the original page, this was the control. And then the new page was the one that just like went horrible. Um, so, so this one actually did okay. Like a versus B, if this one's a, a did okay compared to B, which is horrible. Um, so then we, but then we felt pretty good about this one, which was very host centered. Tell John, that's the host. Tell John who's coming. We put in back, back in those text boxes. We make it look like a form. Got rid of the message field because I just added confusion and it doesn't really make sense if your story is tell the host who's coming because on the previous page you just told the host why you're coming. There shouldn't be another message here. So I got rid of the message. Um, yeah, we cut some other stuff over there just to make it look nice. Um, so anyway, this, if this is a versus B, then we, we did, we launched it and did an experiment. Luckily, like it worked. So like, so like if this is the, this one, then like we totally recovered plus some, um, with that redesign. So like it was like we felt much better after we got to this point because we're like, okay, like this process actually works. Like it wasn't just dumb luck. Um, uh, and, and anyway, so like next stuff started to continue along with this, this host centered framing. Um, because like in hindsight, we didn't think of this at the time, but, but as I was doing this presentation, like, um, if you think about this idea of increasing the number of hosts, you know, increasing motivation or is increasing ability, like, um, putting, making this look like a form, um, and getting rid of unnecessary fields like this paragraph box that, that, that increases ability. It makes it easier. It makes it at least, at least it makes it less like daunting. It's like, oh, I just to fill out a couple of boxes. So that increases ability. But the trickiest part of this, which we already had like this, uh, this was okay. Like that, that was a very like, um, ability was well enabled on this one. But this one's cool because we, we kept the ability, but by leveraging an outside, like an external pressure in this case, the host, we can increase motivation. So like the previous attempts for increasing motivation just didn't work at all. And they were all because they were all like internally focused. It was like, you should do this basically out of the goodness of your heart. Like you should share, you should give this to your code travelers because it's easy and you should do this because they'll like it. But, but that didn't add anything new to the conversation. Speaker 1. Speaker 3. Speaker 2. Speaker 1. Speaker 3. Speaker 3. Speaker 5. Speaker 7. Speaker 1. Speaker 1. Speaker 1. Speaker 11. Speaker 3. Speaker 4. Speaker 15. Speaker 10. This is just a big overview of the process. You can't really read the details. It doesn't really matter because it's super iterative. Research, design, experiment, research, iterate, experiment, design, whatever order you want to put it in. Super iterative. You can see how just in that little sort of condensed, we actually did some more iterations beyond what I presented. But that just shows, especially on the growth team at Airbnb, how we'll just try anything. Like, hey, this sounds cool. We'll do an experiment. We'll learn from it. Do some research if we need to, user research. And then iterate, iterate, iterate. So, wow, you can't even see it at all. It says takeaways somewhere. Anyway, so these are like takeaways. Overall lessons learned. Oh, this is like a process thing. Form strong relationships with all stakeholders to enable fast decision making. So I showed that graph where everything was connected to everybody. And that's not that hard to do. Like, I think it's natural for small companies. Airbnb is a pretty big company now. So I think it's pretty cool that we're able to maintain that. And everybody knows that that's super important at Airbnb. So we try, like, we almost like bend over backwards to get rid of any hierarchy in decision making and empower everybody to be decision makers. A lot of decisions are made by like a five-minute conversation with me. For example, me and the product manager or like a designer and an engineer or whatever. We don't have to, like, always get the product manager or any of our supervisors involved. We just figure it out ourselves. And then we try it. And if it doesn't work, like, nobody gets mad. Like, the fact that that one graph went down horribly. Like, people weren't thrilled, but they were like, okay, cool. Like, we know we can move stuff. So, awesome. Like, that was a good learning. Another thing, like a sidebar. Like, everyone at Airbnb is super positive. Like, super optimistic. It's sort of like a criteria for getting hired. So, like, everyone's just happy. It's weird. You get used to it. Yeah. Okay. So, this, I've been talking about this. Negative changes are okay because they help identify sensitivity. Like, oh, man. Like, we can change things with the design. But also inspire iteration. So, like, when we saw that graph go down, we were like, my colleagues and I were like, oh, cool. Like, we were all just excited to sketch ideas out. And like, this is great. Like, even though it wasn't, like, great, we knew we were, like, pretty confident that we could get it. But, like, worst case, we could go back to what it was. Which wasn't that bad. So, we were pretty confident, though, that we found, we were able to, we could find something that would improve things. Which was good. So, if you have research, or if you are a researcher, this is a good lesson, like, for me personally. Take a second, third, fourth, fifth, whatever. Look at the data, the research data. Like, I showed you, like, the top three things we identified were actually, like, meaningless in the end. They didn't inspire motivation. It was only after we went back. And, like, looked at the data and thought about it a little bit more. Like, what new can we add to this conversation? So, we had this, sort of, had the data all along. And actually, like, if I, I mean, if I, like, to be a little critical. Like, I should have ignored those first three. Because they didn't, they were just, like, tautologies. Like, they didn't, yeah, they didn't add anything. So, anyway, like, research data is great. Go back and look at it if you have it. I talked about this, too. But, like, to increase motivation. A great source of that is external sources of pressure. So, it's really hard to just, like, to internally, like, self-motivate people. I guess it can be done. But if you can find some other source to do it, like, you should do it because of the host. That, I think that just works better for motivation. Yeah, like, this is another thing. Like, this was a, that project was great learning for our team. Because this whole idea of, like, the one thing. There's another, there's a great, there's a couple great blog posts. I think there's a book. It's called the Jobs to be Done Framework, which is pretty similar. It's this idea that, like, like, every page or every screen or whatever has, like, one thing that, like, one main thing that you want users to do. One, one ask, one request, whatever it is. One, one thing that you want to happen. So, you should, like, strip away anything unnecessary and just get it straight to the point. Like, what's the one thing you want to happen on that page? Yeah, so, like, I mentioned before, like, the Facebook. Uh, growth team had, like, sort of developed this reputation. Whether it was deserved or not. And other growth teams have this reputation, too. There's this thing called growth hacking. Which is, like, oh, it sounds so cool. And it's, like, oh, hacking growth. Um, but, like, it has sort of negative connotations. Because it implies that you don't really care about the overall experience. You're just, you're just taking shortcuts to, like, to, um, to, to get, like, an extra 10% signups. But, um, so we do that. We make these tactical changes, like I showed you. Little changes to the experience. But it's always with the goal, with the mindset of improving the overall experience. We never want to sacrifice the overall, like, user experience of, like, the service design, almost, of Airbnb. Just to get, like, 10% more signups. That would be a total waste. So everything we do is, like, are we actually helping users? We don't, we never want to, like, deceive users or whatever. Um, of course. Why, who would want to? There, that's it. Yeah, um, so, questions? Anyone? That's all I have. Yeah. I have one question. Yeah. I'm thinking that trust must be very important to you guys. Yeah, yeah, for sure. Both for the host and for the, for the people who rent the place. Yeah. How do you, I mean, how do you enable that? Yeah. What do you do in your communication? Sure. Yeah, I think, um, one big source of trust is, like, this isn't the best page, but, um, the best example of it. But I think, like, we have, like, amazing visual designers. Like, just outstanding. So, like, I think one of the things we do for trust is just have a very, like, professional, uh, we make everything very professional and polished. And I think that inspires trust, just visual design. Um, another thing regarding, like, the content or, like, what we write. We do a lot of things called copy experiments. Um, so, for example, this one says, tell John who else is coming. Uh, we'll, like, it's in the works to experiment with this and try other, other language right here. And just changing that text. I'm sure we'll, we'll move things up and down a couple percent. Like, like, if we said, you should tell John who is coming. Or, like, John wants to know who's coming. Like, all these little things can, can totally influence what people, how people perceive that. Um, and I think, ultimately, it's trust. Like, if they don't, if they don't trust, like, what you're trying to get them to do, they're not going to do it. So, I don't know. Does that sort of answer your question? Or? Well, yeah, a little bit. But maybe more on the, the big picture. Yeah. Big picture. So, let's say, um, I got to do a park. I'm moving in. I'm, of course, in November. I have an extra room. Maybe I should just rent out this one room. But would I feel comfortable with living with someone for a few days? Or, I mean, how do you create that trust? Oh, like, among potential hosts, for example. Yeah, yeah. Exactly. And also for the people who rent with me. Who knows? Maybe I'm a psychopath. Oh, sorry. Yeah, yeah. Yeah, I was just asking about the trust. Yeah. In regards to Airbnb. Because let's say that I have this apartment and I have an extra room. And I want to rent it out and make some money on it. So, how do. How do Airbnb makes me feel trustworthy towards the people who will visit me? Yeah. And the other way around? Sure. So, I. Most of my research is with the guests. So, that's what I can speak, like, to the most. I haven't done a lot of host-facing stuff. But I know, like. So, some of my colleagues at work work on, like, the host side. So, getting people to list their space. And then, like, try to become hosts. They do. There's, like, a ton of research that goes into understanding, like, what people's potential concerns are. Yeah. Why they're worried. You know. And then, addressing those, like, through either messaging that we have. Like, maybe it's throwing out a statistic. Like, this isn't a real one. And it doesn't even exist. But it would be, like, 90% of the people on Airbnb are just like you. Like, people that are just renting out their spare bedroom. Or something like that. So, this idea of social proof. Which is just saying, like, everyone else is doing it. Basically. That's something that generally works for the guests. And for the hosts. We have this system. Which is, like, down here. You can't even read it. Of reviews. Right? So, like, reviews are one of the most powerful things. Just, obviously, in general. On online platforms to create trust. It's something that we, you know, feel is super important on Airbnb. And also, one more thing for the hosts. Like, hosts can decide, like, they can sort of set their filter for who they want to accept. Or who they don't want to. So, you can, like, you can set it to just, like, friends. Or friends of friends on Facebook. Or there's a lot of control that we give hosts. To sort of, like, I think. Yeah. To sort of, like, ease them into the idea of hosting. If they want to. And, yeah. I don't know. We can talk more. Thank you. That's, like, the million dollar question. How do you inspire trust? I should have done that. Anyway. And Mia will be coming around with the microphone. So, everybody can hear the question. Any other questions? For Chris? Can I? Can I? Yep. This one over there. Thanks. I just wanted to know. In between these tests. Yeah. How long did you wait before implementing a new test? Oh, yeah. Did you wait for an amount of unique users or what? Yeah, yeah. That's a really good question. So, like, people that measure. Teams that are working on booking. They have to run their experiments for a lot longer because of, like, the seasonality. Or even, like, the weekly patterns of bookings. So, like, more people book on certain days of the week. I don't know which days. But for us, because we're focused on signups, we don't have that. Those patterns, really, to worry about. So, we can run an experiment in, like, two days and get all the results we need. And then the time in between experiments is just however long it takes us to redesign and re-engineer the page. Which, I don't know. It could be a couple days. It could be, like, a couple weeks. It depends on, like, priorities and stuff like that. It's generally, like, as fast as we can do it. But, yeah. And there's no A-B split testing? Sorry, no what? A-B split testing? Oh, yeah, yeah, yeah. So, everything we do is, like, A-B testing. Yeah, I just didn't really show that level of complexity. But, like, especially on the growth team, we don't launch anything without testing it. Like, everything is, like, a 50-50 test. Or sometimes it's, like, a subgroup, like a 10-10 test. But, yeah, we're always. We never dare launch anything. Yeah. Thanks. Yeah. I can jump in with a question. I think we have time for a couple more if you have. Okay. You said something in the beginning. You said that growth comes from love. Yeah. The other way around. Is that kind of a company slogan? Together with being happy? Or? It's. I think it's becoming one. Like. And how do you do it? I mean, how? Yeah, yeah, yeah. How do you love? Well, that's, like, a huge question. And, like, there's. So, like, how do you measure? Love is an interesting question. So, like. Like I said before. Like, most of the company is focused on nights booked. Like, that's, like, the metric. That's a business metric. The growth team is worried about signups. That's a business metric. But that loses. That doesn't, like. That's, like, a second or third order approximation for love. So, we actually are. Like, some of my colleagues are working on ways to, like, measure guest love. We call it. So, like, if anything, that's more like the company. I guess, like, that slogan is. I don't know. Whatever. Like, we are focused on love. And trying to, like, cultivate and measure it among users. And trying to, like, put it on equal footing to, like, nights booked or signups or whatever it is. So, we don't just get focused. Too focused on, like, business things. And lose the forest through the trees. In the meeting, you talk about great signups. Great feedback. Great love. It's, like, one of these. Yeah, yeah, yeah. I think. We don't have, like, a formal way to. To talk about it now. But we definitely talk. Always talk about. If not love. Then, like, the holistic user experience. Like, oh, yeah. But we can't do that. Because it, like, ruins the overall. Like, the cool part is. So, all of this stuff was done with, like. It was, like, me. Maybe, like, two designers. And, like, some engineers. And all of us. Especially the design. One of the designers I was working with. She was great. Because she was super, like, protective of the user. The overall holistic user experience. Which I think is important on the growth team. So easy. It's so, like, enticing to just chase the metrics. Which. Which. I don't know. If you don't have a focus on user experience. You might do. So, anyway. Great. Just a question about. You were showing some graphs. When presenting results. I was just curious. Because they're fluctuating at the same time. Yeah. So, is that traffic source? Or is it. Oh, like the one with, like, multiple colors. Yeah. This one? Yeah, yeah. Okay. No, this is just, like, number of. Number of guests. On the reservation. So, this is, like. If there's two people. On the reservation. Three, four, five, six, seven. So, that just means, like. So, why. So, these are higher. Because if there's. This means there's only one. Co-traveler to add. So, I guess, naturally. You would expect that to be higher. Here's. There's two co-travelers to add. Three, four. So, this is, like. If you have to add six co-travelers. The number of people that add all six. Are. I mean. Are lower than the number of people that add one. So, this is just, like. Breaking down the number of users. So, like. So, like. We don't get to. We know that this. Is. This is going to be like this. I mean. It's almost like a perfect mathematical function. The difference between these. We're more focused on, like. The change within each color. As opposed to the differences between the colors. Okay. Hi. Regarding the testing. And A-B testing. What in. What in terms of geography cultures. Do you differentiate your testings. And designs. Yep. For the. For the. For your. For your global markets. Yeah. Yeah. For sure. So. It depends. Sometimes. If we want to do something super quick. And we don't have time to get it translated. We'll just do it. Like. We'll just. Restrict it to. Anyone that's using the website in English. But otherwise. Generally. We always try to do stuff globally. So. Yeah. We'll get it translated. And then it's just like. 50% of the world. Yeah. Does that answer the. Question. So, you don't. Really differentiate. In terms of love. In terms of trust. Yeah. No. Sometimes we do. Like. It depends. Like. If the results are really interesting. Like. Here. This is like. It was. Not much nuance here. It was. Like. Pretty black and white. But like. But if. If. If like. If results are sort of neutral. Or. We can't really understand. Why something's going on. That's like the next level of analysis. That I. I showed that. That slide. With like. All the different people. And the data scientists. Are the people. They're sort of like. Quantitative. Colleagues. Of mine. That would totally dig into that. And say like. Well. Let's see. If it's just a problem in the U.S. Is it like a language problem. That maybe we. Maybe the. The German version of. Of our copy. Isn't working very well. So like. Yeah. Definitely we. If. If called for. We'll definitely do like the. Like those like. Country specific analyses. To see. If we can figure out. What's going on. Okay. Thanks. Mm-hmm. Thank you. A question. Yeah. Regarding the internal. And external stimuli. Yeah. Because that's of course. Very interesting also. Because. What I understand is. First and foremost. You were inspired by. The social identity. Gaining. Process. Yeah. Of people sharing. And you and me. And our buddies. We're going on this trip. We're going to share this information. And we're a team. And then you discover. It's. It's actually. An external source. Pushing them. Towards. Becoming motivated. To. To act. In a certain way. And are still. Are then still feeling. That internally. It's still what. Drives them. But when they have to be motivated. It has to be like. Somebody wants you. To tell their name. And everything. Yeah. Or do you think. There's still the social identity. Aspect of. No. I definitely think. There's still the social identity part. I think it all adds up. If I can go back to this graph. I think. Okay. So like. So like. This is oversimplifies things. But like. Okay. So I guess. I guess in this. We're talking about motivation. So like. Let's say the ball. Or whatever. We're. Let's say we're here on the curve. So we have some motivation. And I think the. Intrinsic motivation. Is. The stuff that. That's already there. Like. You should do this. Because it's convenient. Or because you're. You're a coach. Travelers want it. I think that gets you. Like up here. And the problem is that we. Like. If we just reinforce that. We don't really move anything. It's just like. People stay here. They don't. They don't go down. But they just stay here. It's like. I know I should. But I don't know. But once we like. Added. Added some motivation. So we. We keep the original motivation. So we. We do tell people like. Your coach. Travelers would get a copy of this. But then we add this. External source of motivation. Like you should do it for your host. And I. I mean. Just to use this like. Framework. I think that. Gets us over the curve. That moves us up. Just enough to get us over the curve. For some people. Obviously. We still haven't solved it for everybody. But like. I don't know. Does that answer the question? Yeah. So it's not like. Either or. Yeah. Yeah. It's both. On top of social identity. Yeah. And your builder. For sure. For sure. Yeah. Thank you. Yeah. I think it's pretty inspiring. That you get excited about a downwards graph. You don't hear that too much. And I'm just wondering. I guess. There are plenty of people. Within the company. That wouldn't necessarily. Get that excited. About seeing a downwards graph. Yeah. So. Do you keep that data. Within the project team. Until. You can present something. That's valuable. Or do you sort of. Share it. And if so. That's a great point. How do you make sure. That people understand. Why you get so excited. About that downwards graph. No. Like. Yeah. I think we keep it. Close. Like. We wouldn't. We wouldn't. Show this to the CEO. And be like. Oh. We found a really. Like. Powerful mechanism. For change. Like. It wouldn't work. Like. So. Yeah. We keep it within our team. And luckily. Like. We have. The product manager. Of the growth team. Is. Like. Super. Just. Like. Not judgy at all. So. Like. When this happens. I mean. Of course. He's like. Oh. We should do something about this. But not like. Who messed up. It's not at all like that. Like. It's like. It's like. Okay. Cool. Like. We know we can do that. And then I saw. Yeah. It's definitely keeping it within that. Like. Close knit circle. Like. Like. Like. That network I showed. Like. The PM is definitely part of that. Like. He's like. I mean. We're all just like. Next to each other. And. You know. I think. Everybody sort of felt. When. When this. When it's like this. We're like. It wasn't like. Oh. That research is really bad. Or like. That design is really. Not what we wanted. It's like. Oh man. Okay. We sort of misunderstood the problem. We misunderstood the problem. What can we do about it. In response. Yeah. So I think. I mean. That's key. Just having people that. Are okay with this. With this sort of uncertainty. I think that's a criteria. For. People. At Airbnb. But yeah. Anyway. I don't think I need a. Microphone. I just want to know. If you can see. If there's any slides. Out there. Oh yeah. Get a copy of the slides. I want to see those. On. Yeah. Yeah. For sure. And you can stop me. Like. If you want. In the meantime. Like at lunch. I'll be happy to show. Anybody. And also. Um. The. Videos will be available. On my meter. Thank you. Um. I'm just curious. Do you. Always test on the live site. Or do you. Also test on. Paper prototypes. Or anything. Oh yeah. Yeah. For sure. Yeah. Yeah. Um. No. We definitely do. Like. We always do. Like we have this luxury. Uh. On Airbnb. We have like. Huge traffic volumes. Uh. And. And we have a digital product. So we can. We have the luxury. Of being able to do these. Like live experiments. Which is. Like. Like. Like. It's like. Totally valid data. Like no one can argue with it. But. Before we get to that point. Um. Like I showed you. That Skype thing. I was doing. Like. I think. Like. Sometimes we do. Like in person interviews. And we literally might have. Like paper prototypes. Um. But like. Maybe the digital equivalent. Of that is. Is like. If we're doing something. Remotely via Skype. The cool part about that. About Skype. Is that we can be. International really quickly. We can just call people in the UK. Or whatever. Um. But then we'll show. We'll just show like. Like a. Static. Image. Like. Actually that's exactly what I did. Like. When. When we got these results. Uh. Then. I. Did this. No actually it was. This. I mean. I showed this. Exactly. It was just like an image. Like literally a static image. That I showed to people. Via Skype. And asked. For their. Sort of impressions of it. So it's. If anything it's like. Lower fidelity than a paper prototype. Because there was no interactivity. It was just like an image. That I asked for people's perceptions on. So. Yeah we. We definitely do a lot of. Uh. Research. Uh. Before we launch stuff. Like it's just. Yeah. They go together. Do we have any more questions? Great questions. If no one else has something. I. I have something not related to this at all. Yeah. Could you tell us about the videos on your. Your starting page on Airbnb. Because I've been. The creepy videos of people. Yeah. Yeah. How did. How did they get over that? And what's the idea? How did they what? The. The. On the starting page of Airbnb. You see videos. But people are sleeping. And moving. Yeah. Yeah. How. How did that come about? Yeah. How. Do you know the story behind that? A little bit. Like it was before my time. But like. It was part of the rebrand. So Airbnb did this big rebrand. Like a year and a half ago maybe. And that was like. Part of it. Were these like full screen videos. Like I think it was just. An idea that. Maybe like. We have this. Art department. They're like people. Like graphic design people. Not even UX designers. But like just graphic designers. Like photography. And videographers. And like people who do illustrations. I actually think it was their idea. I don't know. Don't like quote me on that. At all. But I think it was their idea. It was just like something cool. And then. Like it could be engaging. And. They're sort of controversial. Like. Even within Airbnb. Like. I think people think. Oh they're sort of cool. But other people are like. We could do so much. We could put other content there. Than these big like full page videos. That. Of people sleeping. But yeah. Anyway. Good. Okay. Okay. That's it. Is that it? Yep. Thank you Chris. Can we please give him a round of applause? Thank you. A little bit of Denmark. Thank you. Excellent. Thank you so much Chris. Thank you. Wow. So I have three announcements. One is that. Coffee will be in a minute. It's a very Danish thing. We have to have it. Basically. Interveniously. A couple of minutes. And coffee will be ready. And second of all. I'm really sorry about the lighting. It should get better. As soon as the sun. Moves a little bit further along. So. We've done what we could. We'll see. Third thing. Completely forgot. Oh. I know what it is. I found a phone. Anybody missing a phone? Somebody with connections to Simon? Yeah. Okay. Cool. Great. Well. Please enjoy some coffee. And see you in about 15 minutes. Okay. Bye. Bye. Bye. Bye. Bye. Bye. Bye. Thank you. Thank you. Thank you.