Move over typing to search – if you’re not already searching by image then you will be soon enough. FASHWELL’s deep learning image recognition tech means that in the future every image could be shoppable. And that opens up a lot of opportunities for retailers.

We spoke to FASHWELL’s head of business development Caitlin Crawford about the applications for visual search, the challenge of accuracy and dealing with ‘in the wild’ images.

 

Can you describe what FASHWELL does in a nutshell?

FASHWELL is a technology company specialising in machine learning. We do product recognition for e-commerce. Over the last four years, we have developed technology that can recognise products in images and extract data points about those products or connect them to a shoppable product. We really see our mission as bringing in a completely visual experience to retailer websites and apps using this AI technology.

The company first started with a mobile app where every time you liked something on Instagram it would send you a push notification and tell you where to buy similar products that were recognised in the image.

About two and a half years ago Instagram changed the access rights that they made publicly available. They essentially shut down public access to a lot of customer data for example, customer likes. At this time, we pivoted the business and now we sell our technology in a SaaS model to brands and retailers all over the world.

 

 

Why do you think Instagram changed its model?

I think Instagram was on the path to close down all the outside options of monetisation of Instagram content that did not allow for them to have something in return. They did maintain partnerships with really big companies but for the majority of companies like us, in a startup stage it was not very easy to maintain access to the platform.

It was at that point we thought Instagram would come out with a similar product. We thought for sure they would start to monetise images for shoppable content, exactly what we were doing. Even today, they only dip their toe into shoppable content with advertising placements for retailers, brands and influencers where they can make their images shoppable by tagging or through swipe-up, but they still don’t do anything at scale for shopping.

One of the largest premises of our technology is that we’re spending so much time in these visual experiences scrolling through images for hours every day. This is where we are getting inspired to shop things, but there’s still that lack of connection to the commerce piece.

 

Are there any key things that make FASHWELL different from its competitors?

Because we have a history in the social space our technology tends to be really accurate on what we call ‘in the wild images’. These are really challenging images with backgrounds or pictures of bloggers that are walking and wearing complicated fashion products. We have one of the highest accuracies, if not the highest, available in the market for recognising those products and recognising really accurate attributes about those products.

 

 

How accurate is the tech?

It is really challenging to measure an actual percentage in machine learning. This of course varies on the input coming in (the quality of the image) and it heavily depends on the categories.

With something like an Adidas Superstar I would say our accuracy is close to 100 percent, but when you’re talking about a pair of blue jeans it’s not going to be accurate in terms of getting the exact pair, while it will definitely find something similar looking. Because the use cases of our technology are in ecommerce, we’re looking for the most similar product for you to purchase. We’ve trained our algorithms to the best of our ability to recognise fashion and furniture products, so it’s always looking for the closest match but it’s not necessarily a technology that’s used for finding a 100 percent duplicate.

For example we power Zalando and what we see is that over 70 percent of the images coming into Zalando are screenshots from either competitor websites or social media. These are products where Zalando probably doesn’t have the exact product in their catalogue but we want to enable shoppers to find the most similar thing on Zalando because that’s their preferred retailer and the reason they came into the app to search.

 

Are there any other use cases you’ve identified or see coming up?

Our fastest growing product area is our recommendation engine. We have Similar Product Recommendations and Outfit Recommendations. We’re working with a number of really big brands and retailers and have seen really good results from our visually powered recommendations. In fact, every time we test our recommendations versus what may be a more common recommendation like co-browse, we never lose against engagement and revenue.

Let’s take this example. If I’m browsing for a denim jacket for the fall in the 150-200 euro price range and I’m on a big multi-brand retailer I probably am not interested in jackets that are upwards of 500 euros because I’m probably more interested in the price range of the denim jacket I initially clicked on. That’s why we do a lot of filtering and core merchandising in the back-end of our technology that allows for retailers to build a nice cocktail of product recommendation, that works well for their customer base.

All of our technology and solutions are platform agnostic and are used today on both web and mobile. However, if you think of the use case for visual image search, it makes much more sense on mobile because that is where our photos live. We do have a couple of companies interested in having visual search on the desktop, but in terms of best cases it’s primarily mobile functionality.

 

 

Do you think that desktop e-commerce is being left behind by developers and tech companies?

For sure ecommerce is becoming mobile first. People think of mobile first, but they also want to make sure the desktop experience remains as smooth as that mobile experience, so I believe they will still make sure things can be optimised in both places.

We’ve crossed the point where over 50 percent of searches on most ecommerce sites are happening on mobile, so you really need to make sure that the experience is fully optimised for that mobile experience despite purchases not having the same ratio – that gap will start to close more and more. At some point in the not so far future you’re going to see all purchases happening on mobile.

Personally I do a lot of purchases on mobile when it’s something quick and easy that’s approximately a hundred dollars or less. But if I’m going to make a big purchase or something that I need to think a bit more about, like a vacation, I would generally do that on my laptop.

 

Are you able to tell us some of the most exciting projects you’ve worked on recently?

We worked on a really interesting app with Zalando called Wardrobe, available today in Germany. It’s an inspirational shopping app. It’s really cool because it makes a virtual closet out of everything you’ve either purchased on Zalando or things you upload via an image and add to your closet. The app contains a number of inspirational images that are made shoppable and indexed in your closet. So if I see a photo of an outfit I like it tells me how to build it from the items I have in my closet and what items I could purchase to get the same look.

The great part is this app uses several different areas of FASHWELL technology. It is the best example I have seen of a virtual closet and being able to offer a shopping experience through inspirational images.

 

 

Is there anything you can tell us about how you’ve helped your clients?

The best part of visual search is that we see that most of our customers are getting 30 percent more image searches month on month. When you see the Mary Meeker reports that say by 2020 50 percent of searches are going to be voice and image you might not believe it, but what we are starting to see in our data that this is happening.

We’ve been thrilled to see the success of our recommendation engine. We see on average a 30 percent higher clickthrough rate to our visually similar recommendation and the conversion can be 10 percent better through FASHWELL recommendations. Outfit recommendations is gaining a lot of traction as a form of cross sell and complete the look recommendation – our customers tell us that they see a lift anywhere from 5 to 12 percent on the average order value.

 

Can you see this type of thing coming into a store at any point or do you think it’s going to solely remain on people’s devices?

We actually have a partnership with a smart mirror company. The mirror takes a picture using an integrated camera and it makes on the spot recommendations based on what you’re wearing.

Visual search can also be helpful for in-store associates. We work a lot in the fast fashion space and a big global retailer told us that they struggle when customers come in with pictures of items they wish to purchase. As the inventory or in-store positioning of the clothing is changing so often, it’s really hard for store associates to always identify and locate products. We work with retailers so that their associates can upload that image and then also get more information on the product and the stock information. That’s a much more seamless customer service experience than what is currently available in a lot of stores.

 

 

Do you think chatbots are here to stay?

I think they’ll be a fixture for customer service. In general, it’s a much more relevant tool for customer experience and answering simple questions like ‘where’s my package’, ‘can I have the tracking information’, ‘something arrived damaged what should I do’. I still haven’t seen a chatbot that brings an immersive amazing shopping experience.

 

What do you think FASHWELL’s biggest learnings have been since it launched?

I think we were quite early to this market. There are a lot of visual search players out there and I think we got really excited about things early on that are really starting to take off now. Two and a half years ago when we tried to talk to a big retailer about this whole visual search thing it was a challenge, today everyone is interested in the technology and what it can do for their business. Especially because big technologies are putting visual search at the forefront of their products – like Pinterest Lense, Amazon Scout – retailers are also interested in the same experience on their sites.

 

What industry do you think might be next after fast fashion for visual search?

We’re already working in home and decor as that also made sense from a see something, what to buy perspective. The technology also makes sense for the times you need to refill things in your home or when you need to make a recipe and want a really quick shopping list formulated from an image. I think it will really start to change the way people are shopping and gaining access to information in ecommerce.

 

 

What is coming next for FASHWELL? Is there a vision of the future of shopping that you’re working towards?

I think we’re really excited about our recommendation engines and building those out for further merchandising capabilities. We’re looking to automate a lot of our tooling so that it’s pretty much self-service.

We still believe that visual search will be the future of search, and in the meantime, we are also focused on what we can do for ecommerce and improving conversion and bringing a better customer experience. One example today is that most fashion companies lose a big chunk of money on out of stock products, with similar product suggestions this fills a major gap for retailers.

 

Are there any other retail tech companies or retail startups who you’re really excited by?

FASHWELL participated in NY Fashion Tech Lab this spring and I was lucky to meet a number of really awesome women led startups. A few that stand out are Headliner Labs which is retargeting within messenger channels and Shimmy which is educating garment workers to automate their processes and improve and bring AI to the product design path.

Another company I think is taking off is Looklet, who has built smart technology to shoot products. Instead of a retailer having to shoot every single product on a model, they built technology that automates and renders the products on a model and gives the retailers the ability to style as they choose.

Image courtesy of Fashwell

How else is retail evolving? Find out how it’s moving beyond the store. Want to quickly and easily connect with the players kick-starting trends and inventing the future of retail? Find out how you can transform your team’s thinking using Insider Trends’ little black book here.