Disney Tests Facial Expression Analysis for Movies

Back to using facial expression detection and coding.  Not new, Something we also experimented with, in lab settings, but here with new data science doing the analysis.  Some details in Phys.org.

Disney Uses Neural Net Technology to see if you Love its Movies like you Should  By Mark Coppock

Disney really wants to know if you like its movies. The entertainment company conducts copious research to answer that question, and in fact, it has its own division, Disney Research, that’s focused on figuring out ways to make you love its movies even more. Now, the company is applying neural net technology to determining if audiences are reacting as they should to Disney’s latest films.

The information comes via Phys.org,  https://phys.org/news/2017-07-neural-nets-audience-reactions-movies.html  which took a look at some recent work by Disney Research to apply deep learning concepts to figure out how audiences are reacting based on their facial expressions. So far, the methods are having better results than the organization’s usual techniques.

Researchers are using what they’re calling “factorized variational autoencoders,” or FVAEs, to use an audience member’s early facial expressions to predict how that person will react to a movie in its entirety. The data was gathered by using infrared cameras that monitored audience faces during 150 showings of a total of nine movies including Big Hero 6, The Jungle Book, and Star Wars: The Force Awakens. .... 

(Update)  Discussion in Fast CoDesign.   Disney Movies Could be Watching you. ... 

Automating Multiple Streams of Biometric Analysis

I see that iMotions is out with a method that can analytically combine eye tracking, facial expression, ECG and more.   Have not used their devices and methods, but combining streams of data would be interesting.  This is analogous to ensemble methods in analytics.  Does this further need a cognitive, in a true sense, method to combine results?    More here.

Facial Neuromarketing Lab

Facial methods are expanding.  In PRWeb:  " .... G&R has recently opened a neuro-marketing research lab focused on facial electromyography (“fEMG”) and its applications in evaluating emotional response to advertising, websites, and product design. The lab is located in Pennington, NJ, midway between New York and Philadelphia. It is designed to help companies learn about neuro-physiological measurement and the strengths of using fEMG, try out equipment, conduct pilot studies, and commisssion full-service neuro-marketing research. The facility is also available for researchers interested in using the space and equipment to administer studies themselves.  .... " 

Interactive Affective Picture System

Brought to my attention:

" ... With its latest grant, the Prosper Foundation is pleased to continue to support the initiatives of the Applied Neuromarketing Consortium (ANC) at Northwestern University. The consortium consists of leading academics from the Medill School, Kellogg School and the Feinberg School of Medicine. This grant provides access to a new dataset of unique findings based upon questions around the International Affective Picture System (IAPS). It has been integrated with a 4,000 person subset from Prosper's annual Media Behavior & Influence™ (MBI) Study of more than 15,000 participants. The goal is to better understand and use media influence as a means for achieving improved advertising outcomes.

The Prosper Foundation has long supported academic and scholarly work at Northwestern, having worked closely with the Integrated Marketing Communications program since 2004. Together, Prosper and professors at Northwestern have written several books, made countless industry presentations at the ARF, ESOMAR, AMA, WOMMA and numerous other conferences, and written many articles which have been published in prestigious academic peer reviewed journals.  ... " 

Asked to look at this, does anyone have comments?   In particular the International Affective Picture System (IAPS),    I note that this has been used in gaming as well.   Contact me.

Nielsen Doubles Down on Neuro

Roger Dooley in Forbes.  Precisely the phrase I thought of when I read of Nielsen's acquisition of Innerscope Research.  At first surprised,  since more recent results have led to less credibility in neuromarketing methods.  But looking more closely at Innerscope, this integrates a number of less dramatic methods, like facial coding and other biometrics to their tool mix. So there will be some tools in their box when they come to visit or renew their contracts that must certainly be relevant.  Probably also brings clients to Nielsen.  Following.

Facial Emotion Analysis

Well known work by Paul Ekman is covered in the WSJ. " ...  The Technology that Unmasks Your Hidden Emotions ... Using Psychology and Data Mining to Discern Emotions as People Shop, Watch Ads; Breeding Privacy Concerns .... Companies are amassing an enormous database of human emotions using technology that relies on algorithms to analyze people’s faces and potentially discover their deepest feelings. While the evolving technology has many potential benefits, it's also raising privacy concerns .... " .  The big databases analytics aspects of this is also interesting.

Understanding the Face of Emotion

Determining emotion from facial images. A long term research and innovation interest.  Here news of a September talk in Edge HeadCon.  The introductory piece also has some interesting details.  Is this breaching the last outpost of privacy?

" ... My name is Lawrence Ian Reed. I’m a Clinical and Evolutionary Psychologist over at Skidmore College. Today I want to talk about facial expression of emotion, and a question that’s been gnawing at me for probably six or seven years. We've got some answers, and I’m excited to talk to you guys about what they are.

The first questions that I asked about facial expression were "how" questions: How do our facial expressions change when we’re feeling depressed or when we’ve got bipolar disorder, or when we’re being deceptive? I don’t ask those questions any more for a couple reasons. One is that the questions I’m asking now are much more interesting. The other reason is that I felt satisfied with a lot of the answers. I’m going to review some of those questions and talk about how they led up to the questions that I’m asking now, and we’ll see what you guys think about what I have to say. ... " 

More interesting details at the link.

Facial Encoding by Millward Brown

I was reminded about Millward Brown by this article in Neurorelay.   I met with a number of their people at a conference in Poland now a number of years ago.    They write: " ... Millward Brown (global research agency specializing in advertising effectiveness, strategic communication, media and brand equity research), today announced it has carried out more than 3,000 facial coding studies over the last year, an annual growth rate of 700%. This provider of neuromarketing solutions also announced that Affectiva’s Affdex™ facial coding technology is now included as standard in Link .. " 

Sands Partners with NVISO

Just announced.  A natural interaction capability for neuromarketing work.  Surprised it it not more often done. Directly linking facial imaging with neuroscience technology. With emotion analysis. Taking that mobile is a further plus.

Facial Analysis and the Brand

In Mediapost: A good example  of a non conscious detection of brand engagement i.e. 'neuromarketing' capability.  Here using the analysis of faces during stimulus  exposure as opposed to in-brain signals.  Here taken up by some big and savvy clients in CPG.  I have written here about Affectiva here before.  We tested the idea while it was still being developed at MIT.  " ... Affectiva was one of "The 5 Most Disruptive Technologies" at the 2013 Consumer Electronic Show.... "    .... This should be watched.

" ... Almost a year ago, WPP’s commercial testing company Millward Brown added facial recognition and analysis technology to its offering to better assess the emotional impact of ads. It did so through a partnership with Affectiva, a firm that developed a facial coding technique called Affdex.  Now, after a series of trials, major global clients are signing on to the new testing application. Calling it the largest-scale adaptation of facial coding technology in the industry, Millward said two of its largest clients -- Unilever and The Coca-Cola Co. -- will use the technique for all of their advertising testing in 2013.... " 

Brandwashed Read and Reviewed

Just completed a review copy of : Brandwashed: Tricks Companies Use to Manipulate Our Minds and Persuade us to Buy by Martin Lindstrom.   To be available September 20.  Have enjoyed Lindstrom's earlier books such as Buyology: Truth and Lies About Why We Buy.

I will be posting some review snippets of this book here.  I highly recommend every marketer and consumer read it.  It is very much an updated version of Vance Packard's 1958 book: Hidden Persuaders.  Completely updated and containing lots of interesting examples of how marketing manipulates.   In many, but not all cases, he names specific companies and agencies.  Of course today, unlike in the 50s, we have many more marketing channels than just  TV, Radio, Billboards and Print.  The Internet is now here as well.  Shoppers are armed with their own mobile devices, and data about their activities is being recorded at a furious rate.

People who work in marketing or merchandising will know many of the examples mentioned, but I found some I had not heard of.  Lindstrom also commissions some specific studies for the book, such as in the final chapter where he constructs a test of human influencers, using the Morgenson family.   He also mentions  commissioned studies using brain study methods like fMRI.

Since I worked with the design and use of  mock retail innovation centers,  I will start by quoting the book on just such a concept:

" ... Very few people know this, but most major consumer goods companies, including Unilever, Kraft, Pepsico and Coca-Cola have set up 'fake supermarkets' ...  They stock the shelves with their own products and those of their competitors, then late at night  ... they invite people to come and shop.  While they are browsing the aisle, cameras and brain-scanning equipment are measuring what happens in real time while they select and reject various brands and items.  Not unlike in the film  'Minority Report', these supermarkets generally have a control room lined with TV screens on which reps can actually measure the changes in consumers brain waves as they encounter different positioning of products.  Based on this data, the company develops what in the business is called a Planogram, a model showing where each product should be placed to generate the highest sales .... "  

This is mostly correct, and looks ahead as we further seek to understand the conscious and especially non-conscious interactions of shoppers with the shelf and packaging,  using all of their senses.  The store laboratory allows for new retail designs to be both virtually and  physically examined.  The future is here today.

See also Martin Lindstrom's site.  As one review suggests, Lindstrom is very much an insider, and that makes this book all the more telling and interesting.  In today's world, unlike in Packard's, the industry changes very  quickly.  Advances like Neuromarketing, interactive signage, and electronic shelf labels, mentioned in the book, are changing quickly.  The book provides great examples of their use, but cannot hope to keep up with them.  Go to Lindstrom's site, and follow others like this site to keep up to date.