The Center for Data Innovation spoke with Susana Zoghbi, chief executive officer and co-founder of Macty, an online shopping platform based in Belgium which uses artificial intelligence, computer vision, and natural language processing to create new experiences for customers and improve profits for retailers. Zoghbi discussed how Macty uses AI to customize product recommendations and personalize the ecommerce experience for customers.
Eline Chivot: What led you and your co-founder to create Macty?
Susana Zoghbi: I come from an engineering background, studied mechanical engineering, and later on computer science. When I did my PhD in computer science, I was working on AI for processing language and images all together. Because usually, or at least up until a few years ago, you would have one modality, either visual information or language. What we tried to do was to build models that combine both images and language. So, for example, given an image, we were trying to identify or frame what was in the image or in a given piece of text, to then retrieve images that would display the attributes in the text. We process large amounts of information, not only in the forms of language and images but also in all kinds of sensory information.
After I finished my research, my co-founder and I wanted to start a company, and we created Macty to bring these technologies to businesses. We settled to apply them to ecommerce and fashion by interest and because it’s fun—but also for practical reasons. These models in AI and deep learning are data-hungry, and fashion and ecommerce happen to be a space where there is a lot of data online you can easily tap into. Besides, you don’t need to have a specialized background to understand what fashion is about.
Also, we decided to cater to retailers and fashion brands, as two to four people out of a hundred actually buy something when visiting an online store. These are very low numbers. One of the reasons that influences this is that when you go to an ecommerce store, you always have to describe the item you’re looking for, using words. There’s a search bar, which you use to look for the things you want, which can be a cumbersome process: It takes time, and if you might not know how to describe the product you’re looking for. If you want to find an item of a particular color, say orange or blue, the retailer’s use of colors might not reflect exactly or correspond to that orange or blue color you had in mind. This is an example for fashion, but for instance for mechanical parts, that also applies: If something breaks in your house, and you need to replace it, but you don’t know what it’s called, only what it looks like. We enable that type of visual search directly on the retailers’ website. One of the comments we get is that one could always do that on Google by uploading a picture. We provide the same functionality, except that with Macty you don’t need to be on Google, so users can search a retailer’s collection, not only using language but also visual information.
Chivot: Which technologies do you use, and how do you implement them through solutions that empower and benefit customers and retailers?
Zoghbi: To enable retailers to process images, we obviously need algorithms that understand the content of an image. For that, we use computer vision technology, and specifically we use convolutional neural networks. Those are architectures that process images at the pixel level and capture information from those pixels, to extract features. So, for example, you would have layers of convolution—these are filters that you apply to an image, and from those, you obtain features. The first ones you get are basic, such as lines (vertical, horizontal, or diagonal), and you combine them in several layers, to get more complex shapes. Then you’re able to identify more fined-grained visual attributes, which is what we’re interested in. For any shape or object you’re trying to recognize what is visually distinctive: Patterns, colors, textures, etc. Computer vision lends itself very well to understand this.
We also use natural language processing (NLP). For one use case we enable the user to upload an image and then modify the image using language. For instance, you take a picture of a piece of clothing you like, but if it’s not exactly what you want, you can say that you would like it to look a bit more like this other piece of clothing, with longer or shorter sleeves, etc. Because we combine both image and language information, we use NLP techniques to process those two modalities, which allow those types of modifications.
With our solution “Snap to Shop,” users can go to their favorite retailer’s website, such as Zalando or Zara. Normally once you’re on the website, you would type and describe what you’re looking for. If you use Macty, on the search part there would then be a little camera icon which you can click on. It then brings you to an upload screen where you can upload your own picture, something that you’re looking for, and then the system retrieves items that look most similar to it. Another solution is “Complete the Look:” This is useful if you want to see what to wear with items you already have. We analyze the image you upload, and we match it to other products that would go well together. For example, if you uploaded a blouse, we might suggest a scarf, pants, shoes, accessories that would work well. Finally, “Tweak & Pick,” is based on NLP as mentioned earlier: When you have say a dress that you like, and would like to find something similar but would like to tweak it by changing parts of it, or to find it in a different color or texture.
From the retailers’ point of view, as mentioned, the conversion rates are low (two to four percent), and any improvement can already be significant—which is how we can really help. Of course there are various reasons behind low conversion rates: It can be that users only want to browse around without the intention to buy anything—the so-called window-shopping. But one of the reasons is that it takes time to find the things we want. So if you find the things you want faster, and have spent less time browsing, searching, and filtering, you are more likely to buy. And this is where our solution, by making the user experience even just a little better, by reducing the time spent to search and find, can translate into more revenue for retailers. So the benefits include time saving, enhancing and improving the user experience, and increasing conversion rates. But also, we are making the interaction between users and the web interface more human-like: As humans we communicate using both language and visual information, which you have in a brick-and-mortar store, but not necessarily at the same level online.
Chivot: Who is most likely to use your app? Are there any interesting or surprising shopping habits, profiles, attitudes, or behaviors which you were able to identify with the data?
Zoghbi: The most likely users of our app are young and female—perhaps unsurprisingly. But as our system system can be used in brick-and-mortar stores, we have seen during a demo that teens are very enthusiastic about this. They’re happy to play with the system, because of this “fun effect” of being able to interact in a different way with technology. So if their parent starts using the app, kids are likely to want to try it too, by imitation.
Chivot: How is technology changing the ecommerce fashion industry?
Zoghbi: Technology can be applied in many aspects of the ecommerce value chain. In our case, we focus more on the front end. The end user interacts directly with our product—going first to the retailer’s website—but of course there are many applications for technology on the back end. There is forecasting of demand, and now we have powerful models and algorithms that can process language, images, and time series. Particularly in AI you see the accuracy of these algorithms is increasing as we have more training data and better models. We can predict better the forecasting of demand, you can optimize for the supply chain, you can reduce time for instance using robots in warehouses to pick up and pack products. These are applications where retailers see many benefits as well.
Behind the increasing use of technology, there are always objectives to save costs and increase efficiency, because retailers always have this bottom line to worry about. And brick-and-mortar stores evolve in a highly competitive industry, many retailers are struggling. Any optimizations that can save costs or increase revenues are critical.
From a user’s perspective, we are now all connected online and are expecting, as end-users, to have a seamless and consistent experience between the ecommerce space and the physical space and across our devices, laptops or phones. A lot of technology goes into serving those expectations.
The trends for brick-and-mortar stores are a bit gloomy—they are facing steep competition from large online companies, where products are cheaper, and today there’s a broad diversity of choices available for consumers to pick from. But there is still a place for physical stores, because some aspects of the shopping experience remain physical. When you’re buying clothes you still need to touch and feel, and try on to see how they fit. In addition, stores are rethinking their space, adapting and evolving too, offering more entertaining experiences to attract customers, setting up pop-up stores. They are rethinking the experience and what it is that can keep people coming back to stores. There’s a social aspect of it that just goes beyond the act of buying, and physical stores can tap into that as well.
Chivot: How else could applications and systems like yours could be used, to which sectors could they be applied?
Zoghbi: There are many other opportunities to apply the models we’re using to other areas. The technology we developed is not restricted to fashion and can be useful for many other verticals such as by using mechanical images as mentioned earlier, but also furniture, or medical images.
We are excited about developing Macty to help make fashion more sustainable. We are aware that fast-fashion has an effect on the environment. In a distant future, it would be great to contribute to more sustainable practices in the fashion world. We would like fashion become more an on-demand production ecosystem, where only the things that will be bought will actually be made or produced, so as to avoid a waste of resources. Technologies like ours only represent part of the solution, but they bring us a step closer for people to start even designing their own things, selecting those they really want or like.
