The Center for Data Innovation spoke with Surabhi Gupta, engineering manager at San Francisco-based travel company Airbnb. Gupta discussed how Airbnb uses data to improve its user experience and how a travel company has similar data needs to major tech firms.
This interview has been lightly edited.
Joshua New: You applied for your job at Airbnb after using the service to plan a trip and becoming fascinated with the potential of their data. What exactly did you find interesting about Airbnb’s data?
Surabhi Gupta: I’ve always loved traveling and plan a lot of trips for friends and family. There were two problems that I frequently encountered. First, it was hard to pick a destination and second, it is hard to find that perfect place to stay that is right for me. On Airbnb sometimes these aren’t two distinct problems though. There are over a million unique listings on our site and these listings can also determine the choice of where to travel.
At Airbnb, we can analyze search patterns of where guests are searching from, where they’ve searched, and past trips and put that all together to offer destination recommendations. Having an engaged host community offering valuable travel suggestions was also another source of data that is very unique to Airbnb. The potential of making it easier for people to plan trips and enable meaningful experiences was very appealing to me.
New: You worked on developing a system that can analyze all of Airbnb’s listings to provide users with city summaries. How did you determine that users might benefit from this?
Gupta: We started thinking about how we could inspire people to take trips. Industry research shows that only half of travelers today know exactly where they want to go before planning a trip. I was intrigued by the question of, how does someone make a decision on where to travel? What information can we provide that can help with this?
We have a wealth of information in our listing descriptions and reviews that guests leave. We were able to understand why people travel to a place and what they talk about when they leave a review after visiting somewhere. It turns out that if you travel to a beach destination such as Rio de Janeiro or San Diego, most people talk about the beach in the review. People also share where they spend time doing outdoor activities such as hiking or spend the day shopping. Depending on the type of trip you’re looking for, this information can help you pick a destination and influence where you go on your next vacation.
New: Your work background before Airbnb consists of Google and Microsoft—major tech companies. How could a travel company such as Airbnb use data in similar ways?
Gupta: Most technology companies go through this—they build a successful product, their customer base is growing, and they want to use data to improve the experience. There are a couple of different ways to think about this. We look at existing behavior to better understand how people use our site. Are there common pain points? Are people getting stuck somewhere? This is about understanding the existing usage of our product and finding opportunities. For example, we used to have all the filters exposed on the left hand side of the search results page and we found that there were a few that were used much more than others—those are the ones we now highlight. Additionally, on the listing page, we found that the photos had very high engagement. We decided to make the photos much bigger and more prominent on the page.
Like those companies, we try to understand what is unique about our product and then collect and apply useful data to build something that is personalized to our users. That’s why we want to learn why people travel to a location and analyze data on trip characteristic preferences to better understand Airbnb host preferences.
New: You lead the Search and Discovery team working on improving Airbnb’s search algorithms. How do you go about experimenting with something so integral to the user experience?
Gupta: With traditional search, success is determined by someone clicking on a result and finding the most relevant result at the top. With Airbnb’s search, we care about conversion and ultimately about a successful stay. Are we able to match a guest with the listing and host that is right for them?
Sometimes we interview people about their habits before running an experiment to get some early understanding in what we would like to build. Once we are happy that people are responding favorably to the change, we run an experiment where some proportion of people see the current version live on the site and some people the new version. We then see how the two groups perform, whether we are meeting or exceeding our goals, whether people are happy, and so on. We make launch decisions based on the metrics we see from experimentation.
Experimentation can be hard though, and it is important to get it right. In our case, the process of searching and booking can take many days. During this process guests might use different devices to search, thus being exposed to different experiments. In order to get interpretable and accurate results, we need to make sure we are segmenting users in the right way and that there are equal sample sizes. Once the experiment is complete, we need to make sure we model the interaction effects between experiments, and break down the results into meaningful cohorts. We have a whole team dedicated to thinking about how to solve these challenges.
New: Now that you’ve worked at Airbnb for about two years, do you have any major success stories? Or is it too early to tell?
Gupta: There have been a number of successes. I’ve learned the power of investing in changes both big and small. It is important to go back to the basics and understand customer pain points. I’ve learned the value of having a truly cross functional team to solve these problems. At the start of the year, our insights team compiled a list of pain points experienced by our customers and we used that to influence the roadmap for 2015. We’ve had success in the experiments we’ve run both in terms of small user interface changes and large search ranking model changes. There are two particular projects I’d like to highlight—search ranking and destination recommendation. When I first joined Airbnb, we rewrote the algorithm for our search. Now, we have a team that uses a machine learning model to predict the user interactions through our site. I also helped build a personalized experience to recommend locations to users. The location recommendations are personalized based on data about destination and activity preferences. For example, when a user searches for a weekend destination, we can also show them all the best nearby spots and activities.