5 Q’s for Jeremy Stanley, Vice President of Data Science at Instacart
The Center for Data innovation spoke with Jeremy Stanley, vice president of data science at Instacart, a grocery delivery service based in San Francisco. Stanley discussed how data science can help make the notoriously difficult industry of grocery delivery more profitable, as well as how machine learning can help shoppers spend less time in stores.
Joshua New: On-demand grocery delivery seems like a really tough logistical challenge and other companies have tried—and failed—to pull this off due to its thin profit margins. How can better use of data help here?
Jeremy Stanley: Grocery delivery is hard for four reasons. First, customers want the long tail of groceries—the kombucha, the nutritional yeast, the dried enoki mushrooms. They also want perishable goods, be it produce, hot chickens, or frozen ice cream. They want groceries delivered quickly—they have to be there to receive them, and they want to consume them that day. They also want it inexpensively, or else they’ll go to the store themselves. It’s really hard to do two of these four things simultaneously. Doing all four is what makes Instacart so special, and data is a big part of what enables that.
Two key ingredients to our profitability are our basket sizes and how fast we can fulfill orders.
Basket sizes—how many items customers order—matter because a lot of our costs are fixed like driving and delivery, but our revenue scales with the size of the basket, especially partner and advertising revenue. We increase basket sizes by maintaining a huge catalog covering hundreds of retail partners and tens of thousands of stores. We also increase basket sizes by making search intelligent, so it’s easy to find what you are looking for, and by making recommendations throughout the shopping experience to help customers repurchase easily and discover new food.
Our speed of fulfillment is critical as our largest expense is the wages we pay to our shoppers. So our systems must keep them busy and enable them to deliver faster. Our shoppers pick groceries three times faster than consumers, and drive 60 percent less than consumers would. Data is used to help us balance supply and demand to ensure we keep shoppers busy, and improve how we batch deliveries together—a shopper can pick up 5 deliveries and deliver them in a sequence—and route our shoppers.
New: Your team was able to reduce the time it takes for Instacart to fill an order by 40 percent. How did you go about this?
Stanley: First, we automated and optimized how we balanced supply and demand. Early in our history, operations analysts for each region would staff shoppers manually in Excel every week. They would look at what happened in the past, and make manual adjustments to try to keep shoppers busy without having to turn away customers. We knew this wasn’t optimal, and wouldn’t scale as we expanded, so we committed to building an automated system.
At first, that automated system did something very similar to what our operations team did manually. We would look back to the past in every region, and make incremental corrections to staffing to attempt to improve the outcome in the week to come. Initially, this made everything worse. But we stuck with it, and week after week made incremental changes to the algorithm to the point where it began to outperform our previous manual processes. More recently, we have moved to an entirely new and novel approach, where we combine many distributional forecasts with a simulation and optimization engine to arrive at our final staffing levels.
Second, we made our order batching and fulfillment system more efficient. When we began, we would dispatch shoppers in a greedy fashion. For example, sort customer orders by urgency, and dispatch the shopper who could do the first order the fastest, adding on any additional deliveries if you can. That’s not a bad solution, but it’s far from optimal.
Over time, we have decomposed this problem into components, each of which can be solved to near optimality. For example, we can solve a set-partitioning problem to find the combinations of deliveries that can be done efficiently together in space and time. Then we can separately solve an assignment problem to match shoppers with these delivery combinations. By recomputing these solutions every minute as demand and supply patterns change, and waiting to the last possible minute to make any dispatch decisions, we’ve significantly increased the speed our shoppers can deliver at.
New: How does Instacart figure out what items are in what stores and where those items are? What happens if your shoppers can’t find an item, or if an item is out of stock?
Stanley: This has turned out to be one of the most challenging data problems we have. Some retailers have very clean data about exactly where in each store an item is supposed to be found. They also track their point of sale movement data to know what has been sold, and have great inventory data to know what is in the back of the store. But even then, they may not know what has been restocked on each shelf. On the other hand, other retailers know they received a truckload of watermelons yesterday, and they ought to be “somewhere over there.”
We have a team dedicated to ingesting and structuring the data we receive from retail partners, and that data drives our catalog, what we show to customers, and what we show to shoppers who are looking for an item in a store. We also observe every time a shopper looks for an item, and whether or not they found it. We build models to predict how likely a shopper will be to find any given item at any given store in the future, and use that to filter what customers are able to order, and to trigger them to select replacements in advance.
When an item isn’t found, we do our best to replace that item with a reasonable substitute. We make suggestions to the shopper based on what the customer has ordered in the past, and how people have responded to specific replacements. But with millions of products in the catalog, there is a huge search space to cover for replacements. It’s also not obvious to an algorithm that butter isn’t a good replacement for ghee, or butternut squash for spaghetti squash, or beef spicy sausage for vegan spicy sausage. Our customers choose replacements in real time, and interact with our shoppers via chat, so there is a lot that we can do with machine learning and AI to make this even better in the future.
New: Instacart also uses machine learning to help sort shopping lists to shave time off order fulfillment. How does this work?
Stanley: Even if we had perfect information about where each item in a store is located, a traveling salesman solution to routing a shopper in a store is too naive. There are many obstructions in the store, traffic flows naturally in specific directions, and shoppers will prefer to pick produce first and frozen last, both for food safety, and to give customers time to communicate about replacements if need be.
So the approach we have deployed is to model the sequence that our shoppers pick orders in. For example, given 10 items at a specific store location, what item would a specific shopper start with, and then from there, which item would they go to next? We model that for all of our shoppers at all store locations and for all of the products we shop for. Then, for any new order, we predict the sequence that our best shopper in that store would follow, and present the items to our shopper in that order. This has led to significant improvements in shopping speed, especially for larger orders, as compared to our previous methods of sorting shopping lists.
This was a challenging modeling task, because there are millions of products to consider, for tens of thousands of shoppers at thousands of store locations. We ended up using a deep learning model, as we can embed all of these entities into dense representations, and customize the architecture and objective function to account for the fact that at any given point in the sequence, only a small subset of items can be picked next—only those items remaining in the order. Traditional machine learning techniques would have struggled with these issues.
New: Instacart recently published a dataset of 3 million grocery orders from users in 2017 to encourage people to develop learning algorithms that can help predict what people will buy in the future. Do grocery orders have unique challenges than more traditional ecommerce purchases, such as clothes or books, that make it harder to predict what customers will want?
Stanley: Consumers have unique cultural, taste and dietary needs that influence their grocery purchases. But similar differences drive clothing and book preferences as well. Groceries are unique in a few less obvious ways.
First, consumers tend to purchase the same grocery items over and over again. We all have our favorite brands of milk, rice, or pasta sauce. But it’s important for us to anticipate the cadence of their purchases, and that will vary with the season and their usage patterns. We want the items they are likely to replenish to be prominent on our site.
Furthermore, when customers do buy a new product, it can have a big impact on their future purchasing behavior. If a customer changes their preferred pasta sauce brand, that could lead to hundreds of purchases of that pasta sauce in years to come. And so discovery, while less frequent for groceries than for music, is more valuable when it does happen.
Another interesting component is how items are purchased together. The consumer may be planning to make a recipe, or shopping for a specific event. Understanding how likely items are to appear in context with the rest of the order is really important for helping customers to finalize their grocery orders.
We are really looking forward to seeing what the community does with this data!