Spray on Smart Skin

Examined a number of related methods, without success. 

Spray-on Smart Skin Reads Typing and Hand Gestures No cameras, keyboards, sensor gloves, or VR motion trackers required      PRACHI PATEL  Spectrum IEEE

KYUN KYU “RICHARD” KIM/  STANFORD UNIVERSITY

A new AI learning scheme combined with a spray-on smart skin can decipher the movements of human hands to recognize typing, sign language, and even the shape of simple familiar objects. The technology quickly recognizes and interprets hand motion with limited data and minimal training and should work for all users, its developers say.

Besides finding use in gaming and virtual reality, the new hand-task-cognition technology could allow people to communicate with others and with machines using gestures. Other applications the technologists envision include surgeons remotely controlling medical devices, as well as a new modality for robots and prosthetics to achieve object and motion recognition.

The gesture-recognition technology developed so far has relied on bulky wrist bands that measure electrical signals produced by muscles or on wearable gloves with strain sensors at each joint. Other approaches have involved cameras that track human motion and interpret it using machine learning. Those motion-capturing camera systems require images taken from multiple surrounding angles, which means multiple cameras are needed for a single gesture-recognition system. These multicam systems also suffer from the inherent limitations of vision-based sensors, says Sungho Jo, a professor in the school of computing at the Korea Advanced Institute of Science and Technology. Such limitations include regions of a workspace that are not covered by multiple cameras, as well as errors that inevitably occur when a hand or other object is occluded from view.

The software used so far has also been cumbersome. Researchers have typically relied on machine-learning models based on supervised learning algorithms that are computationally intense. They require that a large amount of data be collected for each new user and task, all of which require humans to label.  ... ' 

And More Skin, Now for Health Monitoring

Argonne at work. 

Skin-Like Electronics Could Monitor Health Continuously

Argonne National Laboratory

Joseph E. Harmon, November 16, 2022

Scientists at the U.S. Department of Energy's Argonne National Laboratory, the University of Chicago, China's Tongji University, and the University of Southern California are developing flexible, wearable electronics that can monitor the wearer's health. The researchers created a skin-like neuromorphic chip from a plastic semiconductor film integrated with stretchable gold nanowire electrodes. In one experiment, the researchers assembled and trained an artificial intelligence device to differentiate healthy electrocardiogram signals from signals indicating health problems, which it did with more than 95% accuracy. The researchers also analyzed the plastic semiconductor under X-rays, in order to better understand its structure.  ... ' 

Technology for Textiles

 Lately have had several questions regarding related tech.  Here an overview

Technology Transforms Textiles  By Samuel Greengard,  Commissioned by CACM Staff, November 17, 2022

It is easy to overlook the prominent role of fabrics and textiles in our lives. We wear them, we sleep on them, and we use them to cover windows and floors. Although we've witnessed a steady stream of advances in materials, fabrics haven't changed much over the years.

That is about to change. New types of fabrics are rapidly taking shape. Unlike past smart textiles that were mostly used to track motion or for visual displays, these robotic fabrics sense motion and movements and adapt accordingly. This makes them ideal for use in athletic training, rehabilitation, and prosthetics.

"Soft artificial muscle filaments … can be programmed to generate desired motion, deformation, and force," explains Thanh Nho Do, a senior lecturer in the Graduate School of Biomedical Engineering and Tyree Foundation Institute of Health Engineering (IHealthE) at the University of New South Wales in Australia. "The possibilities in the

"Fiber-like artificial muscles could form the basis of soft robotic exoskeletons," notes  Rebecca Kramer-Bottiglio, John J. Lee Associate Professor of Mechanical Engineering and Materials Science at Yale University. The resulting textiles could lead to adaptive clothing, but also new types of household products and even lightweight, stowable, shape-changing machines, she says.

Living in a Material World

Over the last few decades, technology has become increasingly interwoven with fabrics. Synthetic fibers such as Kevlar and Spandex have changed what we wear—and improved comfort. At the same time, smart casual wear, sportswear, shoes, and even smart business suits have appeared.

Robotic textiles and fabrics reshape the concept. "Advanced fibers and fabrics have the potential to create a whole new platform for experiences through wearables," explains Ozgun Kilic Afsar, a design engineer and graduate research assistant in the Massachusetts Institute of Technology's Media Lab. "These new digital fibers can see, hear, and sense their surroundings; communicate; store and convert energy; monitor unconscious and conscious biological processes like breathing, and even form fibers and fabrics that change their shapes."

Afsar is part of a group that has produced reconfigurable OmniFiber technology using fluidic fiber actuators. The material supports artificial muscle-based textiles that can deliver haptic feedback for breathing. This could benefit athletes as they train as well as individuals recovering from surgery, for example.

MIT researchers are also developing fabrics that incorporate fiber supercapacitors, fiber diodes, fiber transducers, fiber ICs, and memory devices to sense, memorize, learn, infer situational context, and respond to the environment or body with which they're interacting. One design, 3DKnITS, relies on a special type of plastic yarn in a floor mat or garment to sense body motion with 99% accuracy.

At the University of New South Wales, Do and a team of researchers have designed fabrics with tiny silicon tubes, approximately the circumference of a piece of yarn, that are braided and woven into a fabric. Hydraulic pressure stimulation causes the fabric to take on various preprogrammed shapes and forms.

Researchers at the university are also experimenting with more futuristic concepts, such as textile sheets that could grow along human skin and create a smart wearable suit that is always a perfect fit. Do says that such a garment, which he likens to an "ironman suit," could incorporate artificial muscles that could aid rescue workers and create shape-shifting bio-mimicking robots.

"These smart textiles have great potential for many applications ranging from basic fashion to compression garments, wearable haptics, wearable assistive devices for rehabilitation and human augmentation, and soft assistive robots," Do explains.   ... '   (considerably more) 

MIT Engineers Fabricate an Electronic Skin

Always interesting things from MIT.

Engineers Fabricate a Chip-Free, Wireless Electronic 'Skin'

By MIT News, August 18, 2022

MIT engineers have devised a new kind of wearable sensor that communicates wirelessly without requiring onboard chips or batteries. Their design, detailed in the journal Science, opens a path toward chip-free wireless sensors.

The team devised a semiconductor film that can be used for chip-less, flexible e-skins. The material is flexible and breathable, thus providing better comfort for wearers of sensors. The devices convert electrical energy into surface acoustic waves using a piezoelectric resonator. The researchers found they could use the material simultaneously for both sensing and wireless communication.

"Our device could make a system very light without having any chips that are power-hungry," says Professor Jeehwan Kim. "You could put it on your body like a bandage, and paired with a wireless reader on your cellphone, you could wirelessly monitor your pulse, sweat, and other biological signals."

From MIT News   

Vaccination Without a Shot

My own informal conversations with people would indicate that there is a fear factor involved, and anything that would decrease that would increase participation. 

 3D-Printed Vaccine Patch Offers Vaccination Without a Shot

University of North Carolina at Chapel Hill,  September 23, 2021

Scientists at Stanford University and the University of North Carolina at Chapel Hill (UNC) have engineered a three-dimensionally (3D)-printed vaccine patch that offers more protection than a typical shot in the arm. The microneedle-studded polymer patch is applied directly to the skin, with a resulting 10-fold greater immune response and a 50-fold greater T-cell and antigen-specific antibody response compared to injection. UNC's Shaomin Tian said the microneedles are 3D-printed, which makes it easy to design patches specifically for flu, measles, hepatitis, or COVID-19 vaccines. Entrepreneur Joseph M. DeSimone said, "In developing this technology, we hope to set the foundation for even more rapid global development of vaccines, at lower doses, in a pain- and anxiety-free manner."

IEEE Spectrum on Skin Display

A means of convenient and changeable skin display.    Feasible, but is it broadly acceptable?

SKIN DISPLAYS WILL GIVE WEARABLES THEIR INDEPENDENCE

When the display’s on your skin, you can leave your smartphone in your pocket

An elderly woman, living alone in a small mountain village, relaxes in the afternoon after finishing lunch. A square sheet of thin rubber clings to the back of her hand. Like a poultice, it stretches and then wrinkles as she flexes her fingers.

As she reaches for her teacup, the square lights up with a message: “TAKE YOUR BLOOD PRESSURE MEDICINE." She smiles, remembering how she used to struggle and often fail to remember, even though her smartphone had been programmed to send her such alerts. But now, thanks to that business-card-size patch on her hand, she hasn't missed a single dose. Indeed, her blood-pressure value, which she can now see on the display, is well within a healthy range. ... '

Smartphone Camera Can Illuminate Bacteria Causing Acne, Dental Plaque

Diagnosis Application.

Smartphone Camera Can Illuminate Bacteria Causing Acne, Dental Plaque

By University of Washington News, June 17, 2021

Researchers at the University of Washington have developed a technique to identify potentially harmful bacteria on skin and in the mouth using images taken by conventional smartphone cameras.

Their cost-effective approach — which could become the basis for home-based methods to assess basic skin and oral health — combines a smartphone-case modification with image-processing methods.

The researchers attached a three-dimensional printed ring featuring 10 LED black lights around the smartphone case's camera opening, which researcher Qinghua He said serve to "'excite' a class of bacteria-derived molecules called porphyrins, causing them to emit a red fluorescent signal that the smartphone camera can then pick up."

Researcher Ruikang Wang explained, "If you have bacteria producing a different byproduct that you want to detect, you can use the same image to look for it — something you can't do today with conventional imaging systems."

From University of Washington News ... 

Possibilities of Wearable Electronics

With more technical details linked to from the the article below. 

Researchers realizing the limitless possibilities of wearable electronics  by Erin Matthews, University of Ottawa

Bnoît Lessard and his team are developing carbon-based technologies which could lead to improved flexible phone displays, make robotic skin more sensitive and allow for wearable electronics that could monitor the physical health of athletes in real-time.

With the help of the Canadian Light Source (CLS) at the University of Saskatchewan (USask), a team of Canadian and international scientists have evaluated how thin film structure correlates to organic thin-film transistors performance.

Organic electronics use carbon-based molecules to create more flexible and efficient devices. The display of our smart phones is based on organic-LED technology, which uses organic molecules to emit bright light and others to respond to touch.

Lessard, the corresponding author of a recent paper published in ACS Applied Materials and Interfaces, is excited about the data his team has collected at the HXMA beamline. As Canada Research Chair in Advanced Polymer Materials and Organic Electronics and Associate Professor at the University of Ottawa in the Department of Chemical and Biological Engineering, Lessard is working on furthering the technology behind organic thin-film transistors.  ... ' 

Intel Building Artificial Skin

Modeling chips more closely to biological neurons.

Researchers use Intel’s neuromorphic chip to build artificial skin  By  Maria Deutscher in SiliconAngle

Intel Corp. today revealed that researchers are using its neuromorphic chips to develop artificial skin for robots, in a project representing one of the first practical applications of the technology.

Intel, the leading maker of central processing units, is researching alternative chip architectures to help it maintain its long-term competitive advantage. Neuromorphic computing is one of the areas where the company is active. The term refers to an emerging class of chips that have transistors modeled after neurons to help them run artificial intelligence models faster.  ... " 

See also:
Neuromorphic Chips Take Shape  By Samuel Greengard
Communications of the ACM, August 2020, Vol. 63 No. 8, Pages 9-11
10.1145/3403960 ... 

Electronic Skin

 Note the long history of this kind of effort.

Integrate Microchips for Electronic Skin
Leibniz Institute for Solid State and Materials Research (Germany)
January 22, 2020

German and Japanese researchers have developed an active-matrix magnetic sensor system in a step toward the creation of electronic skin. The system is comprised of a series of magnetic sensors, an organic bootstrap shift register as a control mechanism, and organic signal amplifiers. All the electronic elements are based on organic thin-film transistors and integrated within a single platform; the device exhibits high magnetic sensitivity and resilience against mechanical deformation, and can facilitate two-dimensional magnetic field distribution in real time. Said researchers Oliver G. Schmidt and Daniil Karnaushenko, "[The] ultra-compliant and flexible nature of these devices is [an] indispensable feature for modern and future applications such as soft-robotics, implants, and prosthetics.”

P&G: An Inkjet Printer for your Skin

My former employer came out roaring on the stock market today.    See here in Reuters.   P&G raises full-year forecast after beauty, healthcare brands drive profit beat .... 

And also came out with what seems to be a really innovative idea:

Businessweek
P&G Made an Inkjet Printer for Your Face
After a decade of R&D, its Opté beauty wand gives a targeted, diagnostic approach to putting makeup on your skin.

By Tiffany Kary in Bloomberg

Inkjet printers aren’t often ­associated with luxury, beauty, and luminosity. But Opté, a handheld makeup printer for your face, promises to deliver exactly those things. With tiny cameras that take photos at 200 frames per second, Opté quickly scans for blemishes such as age spots and hyper­pigmentation, then covers them precisely as you run the device over your skin. It clicks ­quietly as it goes, shooting out pigment, to instantly camouflage for the day, and also a serum containing niacinamide, a form of vitamin B3 designed to fade imperfections such as dark spots in 8 to 12 weeks.

“It’s about not looking fake,” says Becky Kaufman, a group head in R&D at P&G Ventures, a division of Procter & Gamble Co. in Cincinnati. The target consumer is anyone who wants the look of flawless skin without the appearance of wearing ­foundation—men or women who are looking to have the natural translucence of their skin show through.  ... "

Artificial Skin and Augmented Reality

Links between skin and augmented reality of interest

Artificial Skin Could Help Rehabilitation, Enhance Virtual Reality
Ecole Polytechnique Fédérale de Lausanne
Laure-Anne Pessina

Researchers at Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland have developed a soft, flexible artificial skin, made of silicone and electrodes and equipped with soft sensors and actuators, which provides haptic feedback in the form of pressure and vibration. Strain sensors continuously measure the skin's deformation so that the haptic feedback can be adjusted to produce a realistic sense of touch. The actuators help form a membrane layer and can be tuned to varying pressures and frequencies. On top of the membrane layer sits a sensor that contains soft electrodes that measure the skin's deformation continuously and send the data to a microcontroller, which uses the feedback to fine-tune the sensation transmitted to the user. For now, the scientists have tested the technology on users' fingers. The next step, said EPFL's Harshal Sonar, "will be to develop a fully wearable prototype for applications in rehabilitation and virtual and augmented reality."

Sensor Based Skin for Prosthetic Touch

New advances in touch.   We experimented with devices on the shelf to let consumers touch-engage with a product.    So this might be a way to quantitatively test the experience?

A sensor-filled “skin” could give prosthetic hands a better sense of touch

The “electronic skin,” inspired by the nervous system, can sense temperature, pressure, or humidity. It could be used to give prosthetic limbs a more complex sense of touch.

Humans are amazing: Your body is a sensing machine, thanks to the roughly 45 miles of nerves inside your body that connect your skin, brain, and muscles. A team from the University of Singapore has now used that nervous system as inspiration to create a "skin" for robots that, one day, could improve their ability to detect and understand their environment.

How it works: Sheets of silicon were covered with 240 sensors that can pick up contact, pressure, 
temperature, and humidity. These are able to simultaneously transmit all this data to a single decoder, and should still work when the system is scaled up to 10,000 sensors, according to Benjamin Tee, the coauthor of the study, which was published in Science Robotics today.

What’s new: Flexible robotic “skin” has been tested in previous studies, but this system is the first to enable many sensors to feed back to a single receiver, allowing it to act as a whole system rather than a bunch of individual electrodes, Tee said. Crucially, it still works even if the individual receptors are damaged, making it more resilient than previous iterations.  .... " 

Virtual Makeovers Improving

Worked this space for some time.  L'Oreal has leading the pack.

Virtual Makeovers Are Better Than Ever. Beauty Companies Are Trying to Cash In   By Rachel Metz

Augmented reality (AR) is being pushed to the mainstream by apps in the beauty industry that serve as a consumer tool for trying on makeup. One example is an iPhone app from Ulta Beauty subsidiary GlamST, aided by innovations in underlying technology such as facial-feature and finger tracking. More powerful front-facing cameras on modern smartphones also have helped boost AR's appeal to beauty companies. AR companies said virtual makeup makes sense, given the deeply entrenched consumer mindset of trying on products before purchase. Advocates like L'Oréal chief digital officer Lubomira Rochet said AR is improving sales, with shoppers typically spending more time on an app or website that has AR makeup or skin-care features; Rochet also saw a 10% greater likelihood that those who virtually try on products will buy them, compared to those who do   ... "

E-Skin Functions as Bionic Compass

New idea, with details at the link:

E-Skin Functions as a Bionic Compass in IdeaConnection

An electronic skin able to detect motion relative to the Earth’s magnetic field could have applications in humans and robotics.

The e-skin was created by a team from Helmholtz-Zentrum Dresden-Rossendorf (HZDR) using a thin sheet of polymer foil equipped with layers of magnetic sensors able to detect geomagnetic fields. The resistance of the layers will alter based on their orientation to the Earth’s magnetic field, resulting in a bionic compass that can be affixed to a surface (including human skin). When worn on the tip of a finger, the e-skin was able to detect the direction the wearer was walking—displaying the information on a compass—and also transmit that information to a VR character on nearby screen. ... " 

Procter Olay Uses AI for Skin Advisor

Worked previously with a company seeking to determine physical age via AI classification methods.  Here another.   Note too the long history of P&G with advisory and conversational systems.

In Venturebeat:

AI gave Olay its biggest product launch in 10 years   By Jen Larsen

" ... The company partnered with Nara Logics, and Olay Skin Advisor was born in 2016. It’s a web-based tool, rather than an app, that reduces barriers to entry, Putman says. You take a selfie, and the AI-powered engine determines your skin age and then matches you with a raft of Olay products intended to fix your particular issues — from about 2 million unique regimens, all told.

Olay started out with a very limited pool of data, says Damon Frost, director of beauty and CIO at Procter & Gamble, but it continued to shore up that collection by turning Olay.com into a direct-to-consumer website. That way, Olay could see the direct impact of Skin Advisor, from the number of visitors to the basket size and conversion rate. ... " 

L'Oreal UV Sense

How L’Oréal designed the UV Sense to fit on your fingernail  in DigitalTrends

Beauty tech is still in its infancy, but lifestyle brands are slowly warming up to the idea of incorporating technology to improve user experience and sometimes, the product itself. We’ve seen a smart hairbrush that can determine the quality of your hair, a smart mirror that can identify the health of your skin, and now there’s nail art that can track exposure to ultraviolet radiation when you’re out and about.

The UV Sense is a battery-free wearable electric sensor you stick onto your fingernail. It can measure UV exposure, which you can track via a companion app on your smartphone. It’s from L’Oréal’s Technology Incubator, which has created products such as the Makeup Genius app, where people can try different looks using a smartphone camera, and Le Teint Particulier an in-store device that scans your skin to create custom foundation for you.

The tricky part about the UV Sense is that if people are supposed to wear it 24/7, it has to look good and feel comfortable. We spoke to the team behind this micro wearable to see how they managed the feat. .... "   (pictures and more details) ... 

Printing Electronics on Skin

Researchers 3D-Print Electronics, Cells Directly on Skin
University of Minnesota Twin Cities

University of Minnesota researchers have printed electronics on a real hand for the first time, using a customized, low-cost three-dimensional (3D) printer. Soldiers could use the technology to print temporary sensors on their bodies to detect chemical or biological agents, or to print solar cells to charge electronics. The team also successfully printed biological cells on a mouse's skin wound, in a technique that could lead to new medical treatments for wound healing and direct printing of grafts for skin disorders. The printer uses computer vision to adjust to small body movements in real time during printing, with temporary markers placed on the skin. The technology uses a specialized ink made of silver flakes that can cure and conduct at room temperature. The device is not permanent; it can be peeled off with tweezers or washed off with water. ... " 

Press Release on Olay Skin Care Advisor

More on Olay Skin Advisor:

Olay Unveils Global Skin Analysis Platform Olay Skin Advisor – The First-Of-Its-Kind Application of Deep Learning in the Beauty Industry

Olay’s New Mobile Platform Utilizes AI to Help Women Better Understand Their Skin

"Shopping for skincare has never been more overwhelming, as women are faced with thousands of products and promises"

BARCELONA, Spain--(BUSINESS WIRE)--Today, global skincare brand Olay celebrated its Mobile World Congress debut with the global launch of Olay Skin Advisor, a new platform designed to help women better understand their skin and find the products best-suited to their personal skincare needs. Rooted in a suite of artificial intelligence technologies, Olay Skin Advisor marks the first application of deep learning in the beauty industry, arming women with the knowledge they need to care for their skin and better navigate the often-confusing beauty aisle.

“Shopping for skincare has never been more overwhelming, as women are faced with thousands of products and promises,” said Dr. Frauke Neuser, Principal Scientist for Olay. “Olay’s research shows that browsing the shelf is the #1 purchase influencer for women, yet 1/3 of women do not find what they are looking for. We saw an opportunity to help women understand their skin better than ever, before they even step foot in the store. Our solution is Olay Skin Advisor, which uses artificial intelligence to deliver a smart skin analysis and personalized product recommendation, taking the mystery out of shopping for skincare products.”

- See more at: http://news.pg.com/press-release/pg-corporate-announcements/olay-unveils-global-skin-analysis-platform-olay-skin-adviso#sthash.qQRASriP.dpuf

Selfie Skin Analysis by Olay

An interesting advisor approach done by my former employer using advanced deep learning machine learning techniques.

" ... Procter & Gamble Co. today revealed the launch of a global web platform called the Olay Skin Advisor, which enables a woman to have a selfie analyzed via smartphone to determine the P&G products best suited to care for her face.

The Cincinnati-based consumer goods company (NYSE: PG) said the new Olay Skin Advisor platform, which has been piloted in the United States in beta form, is powered by artificial intelligence technologies.

The Skin Advisor provides a woman with precise, personalized skin education and product recommendations on any smartphone or tablet based on the photo submitted.

The goal is to help women better navigate what P&G described as the often-confusing beauty aisle at groceries or other stores. Buying decisions can be complicated because there are so many products from which to choose.  ... " 

We called this general idea 'virtual beauty'.  See tag below for previous work on that.

And more technical technical AI detail in this piece.

Site of Olay Skin Advisor.

More work on the concept of virtual beauty.