Authors:
William Karnavas, Manpreet Singh Pallan, Tejashri Arote, Fred Boadu
Abstract:
This Disclosure proposes directing robotic help kiosks using sentiment analysis on audio gathered from around a store to direct robotic info kiosks to where they are needed most throughout the day.
It also describes a pipeline for mapping sentiment and confusion throughout a store, which can be used to inform store staff where they are likely to encounter confused shoppers in need of assistance.
Background:
Not finding items where you expect them in a grocery store is a memorable, negative experience for shoppers. Presently, price checkers and item location help kiosks where you scan items for information are a common aid for shoppers, but these are expensive, consume shelf space permanently, and therefore cannot be omnipresent.
Recently, it has become plausible to deploy mobile robots which autonomously and safely navigate an entire store floorplan, assisting shoppers. Researchers and cloud offerings also demonstrate using sentiment analysis to classify spoken words and text transcripts.
Additional problem: if a shopper has difficulty finding an item, nobody helps or assists them, and after some effort they discover the item is currently out of stock, that is a worse experience than simply not finding the item at all.
Description:
This disclosure describes directing robotic help kiosks using sentiment analysis on audio gathered from around a store to direct robotic information kiosks to where they are most needed throughout the day.
It also describes a pipeline for mapping sentiment and confusion throughout a store, which can be used to inform store staff where they are likely to encounter confused shoppers in need of assistance.
Time series data recording sentiment throughout a store floorplan over the course of a day can be stored, animated, and analyzed in retrospect as feedback on the store's design. Using this information, a store can improve its floorplan over time.
The robotic info kiosk would direct shoppers to the location in the store where they can find the desired items. The kiosk may also provide price info about the desired item and a stock check to indicate when an item the shopper is searching for has become unavailable.
See attached pipeline flowchart, which describes the structure of the disclosed pipeline, and the attached mockup of an info kiosk robot based on a robot navigating among shoppers today in Giant stores.
The sentiment analysis pipeline draws audio samples from microphones distributed throughout a store. After filtering out audio where there is no detectable speech, it then performs three kinds of analysis on these streams:
1. Apply speech to text (STT) and perform sentiment analysis on transcripts
2. Apply STT and weight speech samples more strongly if they mention keywords (such as "cannot find", "where is"), and if they mention the names of items sold in the store ("oreos", "eggs")
3. Apply tonal sentiment analysis to speech and record
The end result is a time series of filtered sentiment snapshots, keyed by time and location in the store. This can be used to generate static and animated sentiment heatmaps, and to dispatch autonomous info robots throughout the store. The robots will navigate the store autonomously and safely, avoiding shoppers, and park where sentiment mapping shows discontent about finding items.
TGCS Reference 2676
