Connect with us

David Kepron

Brain Food: Focus on the Face

Why facial recognition systems can help shape store design and customer interaction protocols

mm

Published

on

As far back as Darwin’s assertion that human emotions are encoded into who we are through our evolutionary biology, scientists have sought to determine if the feelings we display on our faces are cross-culturally similar and if we all draw on the same bank of facial gestures to show emotions.

American psychologists Paul Ekman and Wallace V. Friesen developed a system that outlines the way we move our faces in expressing emotion. The original compendium of human facial expression created by Ekman was called the “Facial Action Coding System” (FACS) and included taxonomy of every human facial movement that is tied to our emotional expressions.

All facial expressions can be broken down into various actions, and each expression has been categorized insofar as they relate to specific emotions. When we look at each individual movement, some of them only appear fleetingly across someone’s face as “micro-expressions.” We are able to take note of these micro-expressions subconsciously and combine these perceptions with other body language movements  to decide whether a person we are looking at is happy, sad, angry, etc.

Ekman’s work on facial gestures and emotions demonstrated that there is in fact a list of basic emotions that all humans display. While there is some cultural specificity to their expressions, we all seem to have the movements built into our neural circuitry. Across cultures, there are seven basic human emotions that are expressed by the same facial movements: happiness, surprise, fear, disgust, anger, sadness and contempt.

Great customer experiences are, in large part, reliant on great customer service. The emotional content of the sale is where the money is.

Tuning in to what customers are exhibiting through their facial expressions and knowing a little bit about how their brains are likely to react is not only a key driver to interpersonal dynamics but it’s likely one of the hinge pins upon which customer engagement will come to rely. Being able to tune in to other’s emotional projections is important when the expectation is that sales associates are not just selling stuff in the shopping aisle, but making relationships that grow connections to a retailer or specific brand. This is an increasing challenge for an emerging cohort of young shoppers and sales associates who, according to research, are demonstrating a drop in their capacity for expressing dispositional empathy.

Advertisement

There are a couple of potential paths to follow when talking about facial coding systems:

  1. Facial coding systems can be used to design more attractive products as well as more effective environments that support positive shopping trips.

Companies like San Diego-based Emotient (acquired by Apple this year) have developed and patented “Automated Facial Action Coding Systems” that use video-based expression measurement and sentiment analysis to gauge more than just the age and sex of customers, but also what their facial expressions are saying about their emotional state while in the shopping aisle. In addition to the seven basic emotions determined by Ekman, Emotient’s facial coding systems can determine overall “feeling” – positive, negative or neutral – as well as some advanced emotions such as frustration and confusion.

Being able to detect and record what customers are feeling as they scan product assortments would give retailers data that could determine the effectiveness of a number of design issues, from architecture and store fixturing to environmental graphics and package design. If you can tell that a customer is confused or frustrated while standing in front of a product array, you might gain valuable insight that influences how a customer journey is laid out or how merchandise is displayed.

  1. They can be used to support sales associates in understanding customers and in the development of more attuned customer-interaction protocols.

 In a future state, emerging technologies like augmented reality may allow sales associates to have facial action coding software within a wearable tech device that determines emotional states of customers. This could then provide support to their interactions with rules and tools of engagement. Engagement scripts (i.e., things to remember) and additional information about a customer profile based on shopping history or mined data, may provide sales associates with tools for more effective customer interactions.

If not something being worn by sales associates, then by simply using findings through data captured from customers’ facial expressions while shopping, retailers could better train their in-store teams to recognize shoppers’ affective states and engage them in interactions that promote more positive experiences.

If retailers are looking to defuse a disgruntled customer, empathy is required. If the customer sees defensiveness, anger, defiance or a blank stare, the situation is likely to head down the wrong path. If, on the other hand, the customer sees openness, accommodation and genuine concern for the customer’s point of view, then the situation is likely to defuse in a short time.

Advertisement

Customers want to be heard, not held at arm’s length.

Looking to technology to remedy a lack of skill in interpersonal dynamics is not the fix-all for associate-guest interactions gone awry. Picking up on other’s emotional states is, in part, gifted to us through our collective evolutionary history, but also enhanced if one is able to attune to one’s own internal emotional state. Being introspective and developing the skill to look inward can be an important tool in engaging others at the store level. Sales associates’ being aware of their own emotional projections is an indispensible tool to engaging customers and promoting positive shopping experiences.

Want to have a great set up for a winning experience? Have a smiling greeter or happy sales associate, or even a picture of a happy, smiling face on a big graphic, and you’ll have the customer’s brain primed for a more enthusiastic shopping trip. Activating mirror neurons systems in customers to get them “in the mood” can be initiated by projecting an open, welcoming smile, free of defenses and a bad attitude. Happy shoppers are generally spendthrift shoppers, or at least they are increasingly likely to part with their cash (and even more so with their credit or debit cards) when they are surrounded by positive emotions.

How the face-to-face, embodied interaction happens in shopping places is a determinant of customer satisfaction. In a world where actual visits to the store are only one of the options available to customers – and we can hypothesize that trips to the mall or main street may become less frequent as customers buy from the palm of their hand – retailers bear an increasing burden to send the right message. Everything in the store is important when crafting great customer journeys, but nothing is more important than the connections between the brains of customers and those of the sales associates, or people charged with representing the brand.

The picture customers see in the sales associate’s body language and facial expression telegraphs a message of cooperation and solidarity, or the drawing of a line in the sand. If anger, frustration or sadness is met with anger, frustration or sadness, a feedback loop starts between brains that spirals a positive experience into a vortex of negative emotions.

“Our brain devotes more space to reading the details of faces than to any other object,“ says Eric Kandel renowned neuropsychiatrist and Nobel Prize winner, in an article he wrote for The New York Times in 2013. Being able to read emotions on the faces of customers may be enhanced with facial coding systems, but memorable in-store experiences will, in the end, come down to a sales associate’s ability to extend his or her self in the service of building a relevant relationship.

Advertisement

David Kepron is Vice President – Global Design Strategies with Marriott International. His focus is on the creation of compelling customer experiences within a unique group of Marriott brands called the “Lifestyle Collection,” including Autograph, Renaissance and Moxy hotels. As a frequently requested speaker to retailers, hoteliers and design professionals nationally and internationally, David shares his expertise on subjects ranging from consumer behaviors and trends, brain science and buying behavior, store design and visual merchandising as well as creativity and innovation. David is also author of “Retail (r)Evolution: Why Creating Right-Brain Stores will Shape the Future of Shopping in a Digitally Driven World,” published by ST Media Group Intl. and available online from ST Books. @davidkepron; www.retail-r-evolution.com.

Advertisement

SPONSORED HEADLINE

7 design trends to drive customer behavior in 2024

7 design trends to drive customer behavior in 2024

In-store marketing and design trends to watch in 2024 (+how to execute them!). Learn More.

Promoted Headlines

Most Popular