Computer Vision Analyzes the Store Shelf

We looked at this same problem in a number of ways, but not automated the way this suggests.  Note it has yet to be connected to existing cameras or robotics.

Trax raises $125 million to bring computer vision insights to retailers’ shelves
By Paul Sawers @PSAWERS  in Venturebeat

Trax, a computer vision platform that gives retailers insights into what’s happening on their store shelves, has raised $125 million in a series E round of funding led by Chinese investment firm Boyu Capital. Somewhat oddly, U.K.-based media publisher DC Thomson also participated in the investment round.

Founded in 2010, Singapore-based Trax helps digitize store aisles through image-capturing technology that serves up data and analytics on consumer packaged goods (CPG) on shelves. It may save employees from manually auditing stock for inventory and product promotions, for example. It can also help retailers make product placement decisions by providing a snapshot of what goods are where, and whether they could be shifted to a better position.

The Trax platform ultimately recreates physical store shelves in digital form to guide product and marketing decisions, and big-name clients such as Coca-Cola and Nestle are already using the platform.

While Trax’s technology currently requires capturing images manually via mobile devices, the company said it’s currently working on several pilot projects to provide continuous shelf-tracking via connected cameras that are mounted on coolers or robots and capture images at pre-set intervals. .... " 

Shelf Edge Improvements

Considerable detail, images and video at the link.   A space we worked in for years at Innovation center and in this blog.   Increased visual scale of displays makes lots of sense.

Kroger Grocery Store Shelves Go Digital with EDGE  April, 2018
EDGE (Enhanced Display for Grocery Environment), includes shelves that are 2 inches and 4 inches high by 47 inches wide. It is a digital shelf solution that supports full color and video. The installations are in Cincinnati, Ohio and Northern Kentucky. It consists of 2,500 shelves. The barriers to roll-out were cost, shelf life, customer acceptance and data integrity. EDGE overcame these barriers to create a system that has strong customer and associate acceptance. That customer acceptance resulted in significant sales lift. .... " 

More Cashier-less Stores Planned: New Shopping Data?

Amazon showing some seriousness about the test, with more locations, some word was out there to the contrary.  Now what kind of additional data will this provide to drive better shelf marketing?    These cameras are watching the entire physical selection process.  Trading privacy with convenience?  Comparison to online shopping engagement?  Linking it to virtualized shelf analysis?

Amazon plans to open as many as six more cashierless Amazon Go stores this year
New futuristic convenience stores could appear in Seattle and Los Angeles.   By Jason Del Rey @DelRey In Recode. 

Amazon’s much-heralded convenience store of the future, Amazon Go, may seem like a crazy experiment. But the company plans to open as many as six more of these storefronts this year, multiple people familiar with the company’s plans have told Recode.

Some of the new high-tech stores are likely to open in Amazon’s hometown of Seattle, where the first location is based, as well as Los Angeles, these people said. It’s not clear if Amazon will open up Go stores in any other cities this year. .... "

Aipoly for Autonomous Market Shopping

This will be shown at this week's CES conference in Las Vegas:

Fully Autonomous  Markets

200,000 new small businesses are born every day around the world. We help bring their products on shelves near you using AI, turning any space into a fully working market ... 

No queues, no checkout.

Autonomous Store Platform   Give a brain to your business

Unprecedented Instantaneous DataMarket owners and suppliers can view the movement of  products in real time, gaining instant notifications on stock-outs, sales, and rich analytics obtainable nowhere else.... 

Aipoly's AI can track items on shelves at intervals of a few milliseconds, making suggestions for ideal layouts, allowing fast A/B testing, producing heat maps, and a lot more .... 

No cards, no cash, no checkoutThe Autonomous Store Platform allows you to pick up items, walk away, and charges you upon exit. Download the Aipoly app to set up an account, and you will gain checkoutless access to all Autonomous Markets worldwide. You will get a detailed receipt as soon as you exit the market's door or perimeter.  ....  " 

See Also, the AI Vision App  (More as I get to test it)  Implements shelf recognition and selection.

More on Wal-Marts Bossa Nova Bots among the Shelves

One of my favorite topics.  How will robots be effectively used in real life environments, contexts?  Note 3D imaging.   And how do you get a count of what is behind?  More at the Bossa Nova Tag.

Walmart Deploys Robots to More Stores
By Patrycja Malinowska - 11/14/2017  in CGT 

Originally published by Path to Purchase Institute .... 

A small test of shelf-scanning robots has gone well enough that Walmart is adding upgraded units to 50 additional stores.

The fully autonomous bots employ 3D imaging to dodge obstacles while roving the aisles to check for stock levels, pricing and misplaced items. The robots pass their shelf data to store employees, who then stock the shelves and fix any errors.

Walmart is hoping applying artificial intelligence and robotics will help it combat stock issues, replenish inventory faster and broaden the assortment available to online shoppers while freeing up employees for other tasks.

The machines are from San Francisco-based Bossa Nova Robotics, stand approximately two feet tall and are equipped with a tower fitted with cameras. They are 50% more productive and can scan shelves significantly more accurately than their human counterparts, Jeremy King, chief technology officer for Walmart U.S. and e-commerce, told Reuters.  .... " 

Wal-Mart Testing Shelf Scanning Robots

 Now have seen several applications of store roving and shelf scanning robotics.   The need to get yet better data about the state of retail continues to expand.   Only way you can completely optimize your layout.  Make sure your  most profitable goods are  on shelf.  It will provide better data for more powerful analytics.   See my many posts on the concept.

Walmart is testing out shelf-scanning robots in 50 stores In  Fastcompany, via Engadget

Autonomous robots are taking over Walmart. Well, they are at least taking over more than a few locations. The behemoth retail chain is testing out shelf-scanning robots in more than 50 stores. The devices make sure items are in stock and priced correctly, according to Engadget.

This is all part of the company’s plan to innovate in the coming years. At our Fast Company Innovation Festival this week, Walmart e-commerce division CEO Marc Lore talked about the company’s forays into robotic technology. Beyond these shelf stockers, the company is also researching voice technology to make it easier for consumers to chat with a robot and purchase the things they need. “It will allow them to shop in a very conversational way with a robot … and that robot will know you as well as your mom or dad,” he said.  .... " 

And more with considerable discussion in Retailwire.  With Video.
And in Reuters.

Robot Restocking

Includes a good expert discussion, usually the best parts of these pieces.   Also an approach we did much experimenting with, starting with Blackberry devices.

Can robots keep shelves stocked at Schnucks?  in Retailwire,  by George Anderson

Schnuck Markets, the 100-store supermarket chain based in St. Louis, announced it will run a six-week pilot program at three locations to test robots that will move up and down store aisles to make sure shelves remain properly stocked. The robots will also scan shelves to make sure that each item is in its proper place, aligned with the correct shelf tag.

The robots, which will be deployed three times a day (morning, afternoon and night), will send real-time information to store associates, helping them keep shelves stocked for customers. Each unit, named Tally by the manufacturer, Simbe Robotics, stands 38 inches high and weighs about 30 pounds. The devices use sensors to navigate around the store. The St. Louis Post-Dispatch reports that the units are programmed to avoid busy aisles and stop moving if a customer approaches.

Dave Steck, vice president of IT – infrastructure at Schnucks, said the data the robots collect will also be shared with vendors to help them improve their supply levels to stores. The initial emphasis of the pilot is to see if the chain can improve its stock positions through automation. Later, the technology may be used to reduce pricing errors and address other issues.  .... " 

Alexa Defies and Defines the Moment of Truth

From Wayfind, thought provoking.  No longer a moment of truth, but a moment of speech?  But isn't the cylinder now in control of the conversation?   Sure Brands will still influence the conversation by their value equity, but control is slipping.

Alexa is Brand Agnostic
What happens if brands no longer influence the moment of truth?     By Lee Peterson

Alexa, order a 12-pack of toilet paper, a case of bottled water, and a bag of tortilla chips.

Welcome to the new moment of truth. There is no shelf. There is no cart. There is no register. There isn’t even a laptop or mobile phone. There is only a talking cylinder on a kitchen table. Artificial intelligence has enabled every household to become a store shelf. And in this new, demand-based moment of truth, something is often being left unsaid: Charmin, Dasani, and Tostitos. Or Scotts, Evian, and Santitas. Or even Cottonelle, Fiji, and Mission.   .... "   (read the whole thing) .... 

Real Time Shelf Analytics

In Retailwire, requires sign-in information to get the report.

Aberdeen Study: The Power of Real-Time Analytics on the Shelves
This study, produced by Aberdeen Group, highlights the impact of real-time analytics. See why top performing brands are 50% more likely than others to use real-time technology to empower their back-office with the actionable insights needed to drive operational efficiency, increased sales, and superior customer experiences.

Download the study to see why achieving marketplace performance gains requires that brands empower their back-office with mobile analytics. ... " 

Targeting Physical Out of Stocks

Seen a number of related methods,  this PG article looks at methods that combine image recognition and robotic methods.    We also looked at a method that could use streams from security cameras to count shelf items to address planogram compliance and detect out of stocks.    I liked the idea of multi-tasking.    Further in Progressive Grocer. 

Shelf Scanning Robotics

A little scary in the aisles.  We sought to re purpose security cameras to do this kind of thing.  Also crowdsourced the governance of shelves with cameras. Note the analytics and auditing mentions.

Robot Makes Sure Stores Don’t Run Out of Doritos
A shelf-scanning bot called Tally will help make sure everything is in its place in supermarkets and other retail outlets. ...   The robot, developed by a startup called Simbe Robotics, is the latest effort to automate some of the more routine work done in millions of warehouses and retail stores. It is also an example of the way robots and AI will increasingly take over parts of people’s jobs rather than replacing them. ... " 

" ... Introducing Tally ... The World's First Fully Autonomous  ... Shelf Auditing & Analytics Solution .. " 

Getting Control of the Digital Shelf

Straightforward steps to audit the presence of brands online.  By Schawk.

 " ... To get your share of a rapidly expanding e-commerce market, you need to provide customers with an online experience that’s as true as possible to the product they choose at the physical store shelf. 

This requires that brand owners gain control of the process of e-content development and e-content management. To get started, here are five steps marketers must take to ensure that the brand’s online experience mirrors its offline experience.  ... " 

Connected Products are Next

In Adage: " ... Products as their own media channel ...  " .... But why should anyone in digital media care? Last week at Ad Age's Data Conference, global brands from Loreál to Coca-Cola converged to discuss how IoT-based data (data flowing from and between connectable products, devices, people and brands) can help them build deeper relationships with consumers. While early adopters are now experimenting with creative ways to leverage IoT as a conduit for deeper audience engagement, IoT hasn't reached critical mass yet; IoT discussions still live inside the walls of "digital innovation" and "new product development."   ... "   By Niall Murphy.   Recall discussing this when we were considering only RFID tagging for CPG products on smart shelves.

Magic of Shelf Level Eye Contact

In Supermarket News: Shelf level eye contact can make a big difference.  It was one of the first things we attempted to accurately quantify when designing shelf arrangement.   And this is easier to study now than ever before.  See some of the work on shelf virtualization. But a generalized influence measure is still illusive.

Shelf Eye Tracking

In Progressive Grocer: Report on how people interact with grocery shelves.    We always knew it was more than just about having the eye level shelf.  Its about category too.  And other aspects of shopper behavior

Compliance on the Shelf

I have recently have been looking at applications of compliance. The application to shelf spaces has been a long term interest, see the thread links below.  Retailwire discusses the obvious, that image taking is making this easier than ever, and can be used to understand the physical status of the shelf. Advanced data analysis methods can make these same images efficiently understandable to decision makers.

Shelf Retail Surveillance

In the news today.  Word that a major  CPG manufacturing conglomerate, Mondelez International, is expressing an interest in using cameras on store shelves to gather data about shoppers. Apparently through image analysis, to determine preferences and behavior based on demographics.   First, this is nothing new, it has been done for a decade in laboratory and real shopping environments.  It just has been done primarily in test modes and not in full scale rollouts.   Having large scale databases of video and derived data could lead to useful patterns of shopper behavior.  Those databases,  what is classically called Big Data, have real value.  Privacy is a big consideration.

Note too, that this article and others just show how this company has just expressed an interest in this approach.   All large CPG companies would love this kind of insight.  It does NOT mean they are actively doing it, or are even close to a real experiment.   I have no further information about that.

This blog often covers related topics of retail behavior data and its use.

Large Interactive Public Displays

In IEEE Computer:   An area we looked at for retail presence.  We worked with several companies that used novel light fiber approaches and installed them high on the walls of our retail lab.  The impact could be considerable.  Lower down, on shelf systems could be used to conduct directly the flow of shoppers with messages.  This article is about public displays, but has broader application.  See also the book by 5th-Screen's Keith Kelsen for key insights about display.   Much more on digital signage here.

Wal-Mart Optimizing On-Shelf Availability

Another example, in Progressive Grocer, of how Wal-Mart is active in analytical methods.  " ... A new service from Plano, Texas-based sales and marketing services company Crossmark, is helping manufacturers to better address the challenge of a product’s on-shelf availability at Walmart stores. Called OSA Now, and developed by Plano, Texas-based Crossmark, the solution uses sophisticated algorithms -- customized for each supplier -- to identify store/item combinations that have a high likelihood of having an OSA problem. Typical issues include out-of-stocks, incorrect on-hand inventory, or products that are not on the planogram.... " 

IBM/DemandTec Assortment Optimization

Early in my career I was given the task of how to optimize assortments under the constraints of the market shelf. On the surface the problem looks easy.  You have a set of possible products you want to place on shelf.  You have the value of each of the products to the manufacturer or retailer expressed by profit or historical demand, you have the constraints of space on the shelf.  You may also have design constraints that tell you what products 'should' be placed next to others based on consumer behavior understanding.   Plus a lot of additional and sometimes subtle constraints.  All this can be expressed mathematically, using well known methods that have been known for a long time.  But it turns out the resulting expression is complex and difficult to solve.   We explored how to do this and found it daunting.  So I was very impressed when I first saw an example of how DemandTec did assortment optimization for retailers.  Their method cuts to the chase to increase profitability for retailers based on historical demand while also including the shopper in the model.  They write:

" ... No more guesswork.  The best assortment for each location. The best assortment is the one shoppers expect to see on your shelves, but also the one that delivers the best profits.  So why make guesses? IBM DemandTec Assortment Optimization offers a collaborative approach for defining localized merchandise assortments through a complete understanding of what drives buying decisions – incorporating shopper demand, space, productivity, and profitability. .... Now you can offer items that meet unique demands or targeted shopper segments.  You'll know exactly which items you can replace or substitute for other items in your portfolio.  Most importantly, you'll understand the implications of various assortments on each customer segment. With one common view of the shopper, their decision trees, and the marginal additional value of each SKU to the equation, manufacturers and retailers can work together more effectively to delight shoppers and build customer loyalty. ... "