Aug 19, 2015 · post

Machine Learning Applications in Fashion Retail

This is the first of two articles about recent developments in fashion technology. Part two will focus on implications for consumer privacy.

The next frontier for recommender systems is the retail store. We’re used to associating machine learning with ecommerce giants like Gilt and Lyst, but can data science transform physical stores like Rebecca Minkoff and Zara? The impact would be significant, as more than 90% of retail sales still occur in brick-and-mortar stores, which could use data to redefine in-store personalization.

The introduction of data collection in the retail space does not always require new tools; rather, retailers can gather granular data on customers and products by repurposing existing hardware like security cameras and clothing sensors in new ways. Developments in machine learning have made it possible to auto-process existing video feeds to track customers, from simple movement analytics to full body profiles. Smart mirrors, security cameras (CCTV) and radio-frequency identification (RFID) sensors which create rich datasets that make automated personalization in bricks & mortar scalable.

The first application of new technologies occurs before customers even enter the store. Skybox Imaging and other satellite companies use visual analytics to predict quarterly earnings from parking lot patterns over time. Companies like Nomi/Brickstream, ShopperTrak, and RetailNext, for example, focus on customer traffic analytics before and after store entry. Inside stores, analytics on customer movement inform display layout and product placement, and further quantify where to place featured products. With tools that transform brick and mortar operations into an online clickstream, Euclid boasts the ability to segment shoppers into groups based on an individual’s total number of visits and average shop time. Others anonymize this data. Prism Skylabs, for example, “uses advanced computer vision techniques so that the video feeds it presents back to its customers don’t show any people at all.” Before the interior is even constructed, architects can take advantage of massively improved virtual reality technology to go on pre-construction walkthroughs with IrisVR or InsiteVR. It’s technically possible, then, to collect similar movement data from a focus group of shoppers in advance of a building out new interiors.

Increasingly sophisticated research on body size estimation from a 2D image or video enables CCTV feeds to find humans in-store and, more recently, calculate their body measurements. Smart mirrors, like those used by Rebecca Minkoff, enable customers to request new items from inside a fitting room and checkout on their cell phone. It’s unclear, however, which retailers repurpose smart mirrors to read body data or estimate age from customers. Security cameras, as described by a 2011 NEC Laboratories paper, can find shoppers on video with facial recognition, then analyze their pose and other attributes to classify their clothing into pre-set categories. In late 2014, eBay Research proposed a refined method using clothing detection (identifying only whether objects of interest were present within an image) instead of segmentation (classifying all of the pixels of a given image). Companies like Alvanon are taking steps toward commercializing the automatic creation of body profiles of customers from video feeds with devices selected specifically for this purpose. President Ed Gribbin explains,

“Using a single-scan device we capture data anonymously and we can tell the retailer how many people walked into the store, in what size and how many of those people will fit into their clothing or not,” he explained. “By understanding the size and shape of their customers they will then be able to design clothes specifically for those customers, thereby enhancing fit satisfaction while increasing customer loyalty and retail conversion rates.” 

Because clothing is often produced weeks to months in advance of shipping, data about body measurements improves distribution by matching sizes to local customer body shapes. In the same field of body metrics, Body Labs uses research from Nils Halser and Michael J. Black to translate noisy body scans into full, movable models that are automatically processed and visualized from measurements (currently via license agreement at Brown University and the Max Planck Institute). As their tools are accessible via API, they are affordable for small-scale retailers: a Surface Pro and a single Kinect scanner suffice to create a digital avatar on the spot. If model simulation companies like Looklet expand to cover diverse body types, we might imagine a Body Labs avatar, with realistic clothing and facial features provided by Looklet, moving realistically through an advertisement on a local display.  

The missing piece is automated inventory management, to ensure the right pieces make it to the right zip codes, storage rooms, displays, and fitting rooms in the first place. Most stores still manually request new styles after running low, and physically count in-stock items when taking inventory. By contrast, using data from RFID chips, Zara obtains highly accurate information about inventory in minutes instead of hours, with a much smaller team of associates and less human error. RFID is nothing new, but adapting RFID such that it would not interfere with security sensors or walk away with the customers proved a major barrier for retailers in the past.

For existing inventory, stores can build automated recommender systems that encourage customers to try items that are in-stock, but may not be on display. Early research from a collaboration between Zara and MIT links product performance to the type of sizes available on the sales floor. When the standard sizes are missing (small, medium, or large) and only the endpoints of the size range are on display (extra-small or extra-large), the product will perform worse than a complete set of sizes and takes space from items more likely to sell. In their research, it was recommended for Zara to quickly replenish the standard sizes, remove the item from the floor entirely, or send it off to a store with a more complete set of sizes. That’s why smarter sensor inventory and smart mirrors in fitting rooms are even more important for moving excess inventory - what you see (on the floor) is not what you get. More complete information from improved sensors empowers sales associates to find related items without needing to physically check the stockroom, minimizing customer abandonment of the sale, and helping maximize local inventory turnaround. With companies like EVRYTHNG bringing Internet-of-Things thinking to items with tags as standard as barcodes, it’s possible that some of these benefits of smart tagging are accessible without a major investment in new sensors.

There is already a strong precedent for pulling up unique displays when an RFID-enabled product passes by a screen at Burberry. Like any ecommerce recommender system using collaborative filtering, it becomes increasingly easy to track whether customers wearing item x, of body size y, and estimated age range z are likely to be interested in item a, based on thousands of other customers interacting with item a. Combining the technologies surveyed in this article, retail marketing becomes: reading the body shape of a person near a display, using the RFID tag to identify the product they are interested in, generating a body avatar on the spot, overlaying a realistic simulation of the item onto the model, adding an item similar to something the person is already wearing, and triggering a hyper-personalized advertisement using in-stock merchandise.

The question is, do the privacy risks outweigh the benefits, and do the consumers understand the implications?

- Jessica Graves @sefleuria

Sources:

Business of Fashion [1], Fast Company [1], U.S. Census [1], Nomi [1], Gigaom [1], Genetec [1], Sourcing Journal Online [1], TC2 [1], CB Insights [1], International Business Times [1], EVRYTHNG [1] [2], Deloitte [1], Euclid [1-PDF], IrisVR [1], InsiteVR [1], Northwestern [1], Alvanon [1], Body Labs [1] [2], Brown University [1], Max Planck Institute [1], Looklet [1], Wall Street Journal [1] [2], IBM Developerworks [1], 3D Printing Industry [1], ShopperTrak [1], RetailNext [1], Time [1]