Trying on Makeup Via Live with Amazon

We spent years and much effort looking at the technology options for 'Virtual Makeover'.   Calling  it 'Virtual Beauty'.  Tested it our virtual spaces, and in home settings, never flew strongly.   Always thought the selfie craze would push the uptake.    L'Oreal picked up the idea more strongly.    Here some new stuff via Amazon, likely worth a look.

Amazon's 'Live Mode' lets you try on Makeup via its app

So you know exactly how that lipstick is going to look on you before you buy it.

By Rachel England, @rachel_england in Engadget

Cosmetics giant L'Oreal announced last year that it was buying augmented reality beauty app maker ModiFace. Since then it's deployed virtual makeup try-ons across a number of its own brands, allowing shoppers to test different shades of makeup in a live video or selfie of themselves. Now, the AI-powered beauty technology is coming to Amazon.

From today, Amazon mobile users in the US and Japan will be able to try out Live Mode via the Android app, with iOS slated for launch later this year. Using the front facing camera, shoppers can digitally try on different shades of lipstick, thanks to AI-powered analysis of information provided by make-up brands, as well as images and descriptions found on social media. .... " 

Virtual Makeovers Improving

Worked this space for some time.  L'Oreal has leading the pack.

Virtual Makeovers Are Better Than Ever. Beauty Companies Are Trying to Cash In   By Rachel Metz

Augmented reality (AR) is being pushed to the mainstream by apps in the beauty industry that serve as a consumer tool for trying on makeup. One example is an iPhone app from Ulta Beauty subsidiary GlamST, aided by innovations in underlying technology such as facial-feature and finger tracking. More powerful front-facing cameras on modern smartphones also have helped boost AR's appeal to beauty companies. AR companies said virtual makeup makes sense, given the deeply entrenched consumer mindset of trying on products before purchase. Advocates like L'Oréal chief digital officer Lubomira Rochet said AR is improving sales, with shoppers typically spending more time on an app or website that has AR makeup or skin-care features; Rochet also saw a 10% greater likelihood that those who virtually try on products will buy them, compared to those who do   ... "

L'Oreal Launches AR for Make Up

Again an area we experimented with for years, its seems to now be expanding.  Impressive and interesting details.  Makes a point that no App is required. This topic has been often mentioned here, tag below.  Lot of images at the

L’Oréal Paris Launches New AR Tool For Trying Out Make Up  in VRFocus
L'Oréal Paris and Modiface lets customers test make-up looks in AR with no additional apps required.

A number of companies and brands have begun using augmented reality to allow consumers to try out products and see how they might look in the home or about your person. There have even been a number of make up brands which have launched virtual try-on apps for testing out new looks, however most of them need the user to download a separate app. The new tool from L’Oréal Paris eliminates this requirement.

L’Oréal Paris have used facial recognition technology from ModiFace along with AR product simulation to allow users to discover and experiment with different make up and beauty products using only a web browser. ... " 

L'Oreal on Advanced Digital Experience

Much more at the link regarding what they are thinking of.   Note the mention of the use of voice assistants.

 In Think with Google:

“New technologies, such as voice, AI, AR, VR, are reshaping the way our consumers discover, try, experience, and buy our products. The game won’t be the same when you simply ask your voice assistant to buy the best mascara on the market for you. … Or if you are able to watch a makeup tutorial directly on your face with a virtual try-on. The brands that master these experiences will be the ones people choose.” .... ' 

— Lubomira Rochet, global chief digital officer, L’Oréal.

Augmented Beauty by Modiface at L'Oreal

An area we did lots of research and development on.  Now based on this piece, it seems that the tech has finally caught up to the needs.   But will it practically work as a marketing, sales and operational tool?  Remains to be seen.   See images at the link.

L’Oreal acquires Modiface, a major AR beauty company
By Ashley Carman @ashleyrcarman  in TheVerge

L’Oreal announced today that it has acquired Modiface, a company that’s had a hand in the creation of many custom augmented reality beauty apps, including those from Sephora and Estée Lauder. L’Oreal didn’t disclose the amount spent, but it did tell Reuters that it now owns Modiface’s numerous patents that help users visualize makeup and hairstyles on themselves. The partnership makes sense in that Modiface has already worked with L’Oreal multiple times, including on the launch of its Style My Hair mobile app, which lets users try on different hairstyles. For that app, Modiface manually annotated 22,000 facial images to create the experience.  ... "

Virtual Beauty in Augmented Reality

We spend some time looking at Virtual Beauty applications.  Was never successful in our day.   What can make it work here?   Is this an example of AR success?   A review.

MAC Virtual Try-on Mirror review
It's crazy how well MAC's Virtual Try-On Mirror can mimic makeup     By Brenda Stolyar in Digitaltrend

Feeling completely lost with makeup? MAC’s Virtual Try-On Mirror lets you try on various looks without making a mess.

Shopping for makeup can be a little overwhelming. While we do have social media and YouTube creators to thank for the influx of beauty gurus and makeup tutorials, it’s common to think you won’t be able to pull off stellar and dramatic looks. But what if all you needed was augmented reality (AR) to give you that extra boost of confidence? MAC Cosmetics’ new beauty mirror gives users the opportunity to try on different makeup looks in real-time, with the help of ModiFace’s AR technology ... "

Virtual VR-Like Makeover from Sephora

Was involved with tests of this concept a number of times, it seemed the idea thing for the smart home, cosmetics aisle.  Several makers had this available, but it seems not to have stuck.   Good video at the link,  This uses a more VR-like approach, which is nicely demonstrated in a Sephora store.

How does this look? Be your own virtual make-up artist  In the BBC.
French cosmetics chain Sephora has introduced technology that enables its customers to try on make-up virtually and test out different colours and shades before buying.

Its "virtual artist" tech is also available as a mobile app.
Like other retailers competing in a fast-moving digital landscape, the company is using online tools to engage customers in its physical stores. ...  "

New Kinds of Assistant Magic Mirrors

We examined the Mirror idea under many conditions.  For cosmetics, eye-ware, clothing.  At home and in the store.  The somewhat new idea then is having trained associates guide its use.  A Virtual assistant.  Does not save the cost of expensively trained labor problem.  Saw a similar thing being done in Asia, where having live demonstration people in the aisle is already common.   But will robotics ultimately take that function? See the Lowes' bot example.

Having a system that can really operate well at home has different challenges,  to make it work and be engaging.   The phone is the natural channel.  We tried permanently mounted mirrors.  Has challenges similar to developing sales chatbots, but with image recognition as a sensory input.   I see this has a limited potential for high end products.   But many will still want to see such products live. Could drive some to stores for a demo.

Mirror, mirror on the wall, will you leave me any shoppers at all?

This magic mirror could be a great sales tool. More likely, though, it will just siphon sales to an online rival.   By Evan Schuman, star Influencer, Contributing Columnist, Computerworld  ... 

The interplay between store associates and in-store technology has always been a delicate balancing act. When the tech helps the associate be an all-knowing partner to the shopper, it's a great thing. But when the tech is deployed so that the associate seems to just get in the way, it can ultimately undermine the in-store experience. Enter the HiMirror.

Retailers have toyed with magic mirrors before — it seems to be the retail tech idea that never dies, despite the fact that it rarely works long term — but this mirror goes much further. ... " 

Amazon Outfit Compare

We worked on a number of related virtual comparison ideas over the years.    Aimed at things like eyeware and cosmetics.  None were completely successful.   Can Amazon crack this by funneling their large customer base through this?   What is the conversation needed?

Amazon will now tell Prime members what to wear via a new “Outfit Compare” feature  by Sarah Perez (@sarahintampa)   In Techcrunch: 

Amazon has been steadily pushing its way into fashion over the past several years, with investments in its own private labels – from workwear to activewear – plus increased fashion ad spending and even its own trend-obsessed TV show, Style Code Live. Now the online retailer is looking to dole out its fashion advice to the masses, too, through a new feature called “Outfit Compare,” which is currently available to Prime members.

In the latest version of the Amazon shopping app, Prime members will find “Outfit Compare” in the sidebar navigation under the “Programs and Features” section.

This is the area where Amazon often rolls out its newer services and offerings, like Restaurants, Launchpad, Watch & Shop, Dash buttons and, more recently, its curated recommendations storefront dubbed “Interesting Finds.” ... "

Selfie Skin Analysis by Olay

An interesting advisor approach done by my former employer using advanced deep learning machine learning techniques.

" ... Procter & Gamble Co. today revealed the launch of a global web platform called the Olay Skin Advisor, which enables a woman to have a selfie analyzed via smartphone to determine the P&G products best suited to care for her face.

The Cincinnati-based consumer goods company (NYSE: PG) said the new Olay Skin Advisor platform, which has been piloted in the United States in beta form, is powered by artificial intelligence technologies.

The Skin Advisor provides a woman with precise, personalized skin education and product recommendations on any smartphone or tablet based on the photo submitted.

The goal is to help women better navigate what P&G described as the often-confusing beauty aisle at groceries or other stores. Buying decisions can be complicated because there are so many products from which to choose.  ... " 

We called this general idea 'virtual beauty'.  See tag below for previous work on that.

And more technical technical AI detail in this piece.

Site of Olay Skin Advisor.

More work on the concept of virtual beauty.

Smart Skin Mirror

Links to some of our smart mirror and virtual beauty innovation efforts, but emphasizes skin.  Useful calibration?   In Engadget: 

The first smart mirror you can actually buy focuses on your flaws   
By Cherlynn Low  @cherlynnlow 

Daily photos of your face are meant to flag problems with your skin.

Staring lovingly at your reflection in the mirror gets old, especially if it's not a magical one that tells you you're the fairest of them all. But what if all that time you already spend in front of a mirror could actually make you the most beautiful person in the land? Many smart mirrors already make this promise, but HiMirror is the first one you can actually buy, and it costs $189. The secret sauce comes in the form of a camera that captures your au naturel visage to analyze your complexion, before offering you tips on how to get better skin. I tried it out recently and was impressed by how many things it could do.

The 5.5-pound mirror is more cutesy than the futuristic thanks to its pink accents. It houses a 14-inch LCD screen and has a camera sitting in the middle of a ring flash, which helps light your face for better images. HiMirror snaps a picture of your face every day to analyze your wrinkles, dark spots, dark circles, red spots and pores. The idea is to track how your skin changes over time, and flag your so-called problems so you can take "early measures to improve upon your skin and achieve your beauty goals."     ... ' 

Augmented Beauty

Another look at an old idea.  Been connected with a half dozen attempts.  Will augmented reality tech finally make this work?  See my tags below for more examples.

Augmented Beauty: MakeupPlus Allows Users To Try New Looks In Advance  Instantly try on a new look with this beauty product application.   By Kevin Eta, 

We’ve already had one use of augmented reality (AR) today with Vuzix’s Blade 3000 Smart Sunglasses but what about an AR assistance not in the workplace but for your personal look. That’s the aim of a new AR application from developers Meitu called MakeupPlus that will allow people to try on new looks without even having to reach for the concealer. ... " 

L'Oreal Virtual Reality

We also tried lots of virtual reality in beauty ... but it was never very successful.  Will better quality result in better results?

L'Oreal tries virtual reality.

" ... Animators spent decades trying to make hair look fully realistic, until Pixar created the software used in such movies as Monsters Inc. and Brave. Now that same problem has to be solved in virtual reality.

"From our perspective, hair is the hardest possible problem," explains 8i's Linc Gasking. 8i specializes in modeling fine human details like eyes and fingers, but its team found hair to be particularly challenging, especially the frizzy kind.

Gasking was finally able to crack the hair problem, thanks to a partnership with L'Oréal that's resulted in a new training program for salon stylists affiliated with its professional hair care brand, Matrix. "We're not talking about computer-generated hair," he says. "It’s photo-realistic hair that you can see from any angle."

The project, unveiled today at the Fast Company Innovation Festival by L'Oréal's vice president of Innovation and Entrepreneurship, Rachel Weiss, will roll out to stylists through L'Oréal's "Matrix Academy," an educational program affiliated with its Matrix beauty brand. Instead of traveling across the country for trainings (which are highly costly), or watching two-dimensional YouTube videos, stylists will have the option to learn new styling techniques with a VR headset. ... " 

Consumer Brands Using Chatbots

I have in the talked some of our efforts about using chatbots for consumer brands.  FastCompany more generally has an examination of current brands that use chatbots, including Sephora.  Good general design ideas and cautions.

Very Human Lessons From Three Brands That Use Chatbots To Talk To Customers ... Seamless interactions make for happier and more engaged users who want to keep coming back. ... " 

Amazon in the Bathroom

In Retailwire:   The next frontier, the bathroom, in all its functionality.  In the Smart home we examined many aspects of this, most notably the testing of 'smart mirrors' for both augmentation using approaches like virtual makeovers, and providing information interaction during visits.

" ... Identifying and capitalizing on previously unexplored consumer behaviors seems to be a go-to strategy for Amazon these days, as the company rolls out new offering after new offering to see what sticks. With its Amazon Lockers, its one-hour delivery and its talking in-house personal shopper, Amazon Echo, to name a few, Amazon is attempting to further engrain the company into peoples' habits and daily schedules.

Amazon now appears to be going where few other businesses have ventured. In December, the company revealed plans to address a consumer behavior that, while uncomfortable for some to acknowledge, is an unavoidable reality of the information age. ... " 

Computing What Looks Good and Fits

Have been involved in virtual makeover projects in retail since the 80s.  And have seen attempts to take this beyond just how things look, to see if we can understand if a piece of clothing will fit via some virtual model. A look at a number of Apps and applications that do this.   At very least reducing the expensive process of returns.  And improving the level of consumer satisfaction.

In CACM:   ' ... "Creating a digital relationship with a customer ... could theoretically influence a large share of their consumer spending."  .... Technically, all of that is possible today, he says, but businesses have to figure out how to get there. "It will be a mad race to that point because there is so much to be won—or lost—in this race, creating a digital relationship with a customer that could theoretically influence a large share of their consumer spending."  ... ' 

Smart Interaction to 'Fit' Products

In Retailwire: Same work covered in another recent post.  Another piece on smart fitting rooms to engage with consumers.  Has also often been used for virtual makeover type applications.  Here mentioning work with Nordstrom's.     I was involved with several tests of related concept projects, which provided sufficient interest for new product introduction, but not for ongoing marketing.

L'Oreal and Google Glass

In Engadget:   Reminds me of work done a number of times with virtual makeover.   And related work directing complex maintenance tasks.  " ... The hair care giant (and purveyor of giant hair) just announced Matrix Class for Glass, which gives clients and beauty school students a stylists-eye view of your head. The three-part program includes a video series of in-depth beauty tutorials shot with Google's wearable; Matrix Eye for Style, an "exclusive" salon experience provided by George Papanikolas, who will record sessions with the headset; and a series of lessons for beauty professionals given by be-Glassed hair care superstars. ... " 

Intelligent Color From L'Oreal

In Clickz:  New ways to market cosmetics in context.   With similarity to some of our virtual makeover work. " ... In a location perhaps better known for rats and urine, cosmetics brand L'Oreal Paris is bringing beauty in what it calls the first-ever intelligent vending experience in the New York City Subway, the L'Oreal Paris Intelligent Color Experience.
Launching November 4, the installation works as a three-step process that detects the colors in a woman's outfit, picks the most prominent and related palettes, recommends L'Oreal Paris products and allows women to purchase those products on the spot, the brand says. ... " 

Free P&G Virtual Beauty App

This was something we explored a number of times, starting with CD Rom based delivery, then from the Web, then including in-mall camera kiosks.   Has the time of virtual beauty applications finally become interesting with the Smartphone App? None were very successful, due to a combination of technical, user interest and value-measures.  Here is another try: P&G Offers Virtual Beauty.  " ... P&G Beauty & Grooming brands Clairol, COVERGIRL, Olay and Pantene have joined forces to create a comprehensive, multi-faceted beauty application that provides consumers with an exclusive virtual beauty experience... "   More detail and press release.  For iPhone and Android.