VR Suit Could Help You 'Feel' Things in the Metaverse

Talked this from a customer perspective.

VR Suit Could Help You 'Feel' Things in the Metaverse

By EuroNews, June 26, 2023

A test subject in the immersive reality suit.

The suit's artificial muscles feel similar to human muscles and send haptic feedback to the suit so the wearer can ‘feel’ the virtual environment.

Credit: ETH Zurich

Engineers at Switzerland's ETH Zurich constructed a full-body tactile suit to amplify immersive virtual reality (VR) experiences.

ETH Zurich's Maximilan Eberlein said the Meta Suit's sensors and hydraulically amplified self-healing electrostatic actuator (Hasel) muscles are designed to make VR interaction more natural.

The muscles contract and expand with an electrical current to transmit haptic feedback so the wearer can "feel" the VR environment.

Said Eberlein, "You have pouches that are filled with a special fluid and you spray electrodes on them and you apply a voltage on these electrodes so that these muscles zip together and this leads to a contraction of the muscle.

From EuroNews

View Full Article - 

Virtualizing the Sense of Touch

Researchers Develops High-Resolution, Wearable Electrotactile Rendering Device that Virtualizes the Sense of Touch

City University of Hong Kong

October 20, 2022

A wearable tactile rendering system developed by researchers at the City University of Hong Kong (CityU) and the Robotics X Laboratory at China's Tencent mimics the sense of touch with high spatial resolution and a rapid response rate. The thin, flexible electro-tactile actuator can generate pressure, vibration, and texture roughness in high fidelity. Its operating voltage is less than 30 V due to the use of a high-frequency alternating simulation strategy instead of direct-current pulses, and spatial resolution of the simulators is increased by rendering tactile sensations between physical electrodes instead of at electrode locations. The system could be used in Braille displays, virtual and augmented reality applications and games, and for users of thick protective suits or gloves, like astronauts, firefighters, and deep-sea divers.  Detailed

The Simulated Kiss in VR

 New Experiences in VR 

VR Researchers Have Basically Figured Out How to Simulate the Feel of Kisses

A modified VR headset can create the sensation of touch, either on a user's lips or even inside their mouths.

By Andrew Liszewski  in Gizmodo

Without adding any hardware that actually makes contact with the wearer’s face, researchers from Carnegie Mellon University’s Future Interfaces Group have modified an off-the-shelf virtual reality headset so that it recreates the sensation of touch in and around a user’s mouth, finally fulfilling virtual reality’s inevitable one true purpose.

Aside from handheld controllers that occasionally vibrate, most consumer-ready virtual reality devices ignore senses like taste, smell, and touch, and instead focus on visuals and sounds. It’s enough to make virtual reality experiences far more compelling than they were decades ago, but not enough to truly fool the brain into thinking that what your eyes are seeing is possibly a real-life experience.  ... ' 

See also:  https://www.figlab.com/research/2022/mouth-haptics

Grabbing With Feeling

 Deeper sensors for haptic interaction.

A Flexible Way to Grab Items with Feeling

MIT News, Rachel Gordon, April 15, 2022

The Massachusetts Institute of Technology's Edward Adelson and Sandra Liu have developed a robotic gripper with flexible fingers that can manipulate objects using touch sensors that can equal or surpass the sensitivity of human skin. The gripper employs two fin ray fingers, and the researchers hollowed out the inside to fit a camera that faces a layer of sensory pads made of silicone gel affixed to an acrylic sheet. The sheet is attached to the plastic finger piece at the opposite end of the inner recess, and the finger will seamlessly enfold an object. By measuring the silicone and acrylic sheets' exact deformation, the camera and accompanying algorithms can compute the object's shape, surface roughness, and spatial orientation, plus the force being applied by, and imparted to, each finger. ... '

Hands Free VR

 Note haptics (touch) enabled/

Physical Sensations in VR Go Hands-Free

IEEE Spectrum, Michelle Hampson, April 11, 2022

A new virtual reality (VR) haptics system developed by researchers at the Chofu, Japan-based University of Electro-Communications (UEC) eliminates hand-based hardware by manipulating the forearm instead of the hand to generate physical sensations. The lightweight system features an external sensor camera that tracks the user's finger movements, and applies haptics sensations to the top, bottom, or sides of the forearm to match those movements. "We were surprised that even with this new haptic presentation method, we were able to obtain a high comfort level without any training time," said UEC's Taha Moriyama. Users described the feeling of the new system as "symbolic" of moving a VR object, rather than feeling like they were truly grasping one.  ... ' 

Prosthetic Hands and a Sense of Touch

New and better sensory Haptics

Touchy Subject: 3D-Printed Fingertip 'Feels' Like Human Skin

University of Bristol News (U.K.), April 6, 2022

Researchers at the U.K.'s University of Bristol have developed a three-dimensionally (3D)-printed fingertip that could improve robotic dexterity and the performance of prosthetic hands via an in-built sense of touch. The artificial fingertip incorporates recordings of the human tactile sense using a 3D-printed mesh of pin-like papillae on the underside, which mimic those found in human skin. "Our 3D-printed tactile fingertip can produce artificial nerve signals that look like recordings from real, tactile neurons," explained Bristol's Nathan Lepora. He said the fingertip was able to "feel" the same ridged shapes used by psychologists, which scientists employed in 1981 to plot electrical recordings from human tactile nerves.  ... ' 

A Revolution in Haptics

 An area we worked in within innovation labs with consumers, how do products feel in context?

A Revolution in Haptics  By Paul Marks Commissioned by CACM Staff, February 3, 2022

Force-feedback devices like haptic gloves and gaming vests, which are peppered with electrically driven actuators, let people feel they are actually touching, or being touched by, three-dimensional (3D) objects in virtual reality (VR).

Making 3D visuals tangible does not always have to involve the use of such complex wearable technology, say three research groups that have come up with some intriguing – if not downright bizarre – new ways for people to interact with virtual environments.

Alternative haptic methods are needed, says Pedro Lopes, who leads the Human-Computer Integration Lab at the University of Chicago, because haptic gloves and vests are bulky, power-hungry and hard to build force-feedback motors into. "So people in our field, and in the haptics, VR, and AR (Augmented Reality) industries, need to bend the laws of physics and propose new types of actuators that consume less power and which are easier to integrate."

In attempting to engineer such a system, Lopes and his Ph.D. students Jasmine Lu and Jas Brooks have come up with a pretty radical idea: their new take on haptics applies a variety of chemical stimulants to a user's skin to evoke sensations related to the content that's playing out in a VR story or a game at any time.

To apply their stimulants, the team developed a prototype wrist-worn silicone sleeve, and a cheek patch for facial application, which use micropumps to apply to the skin topical skin-safe dosages of one of five chemicals, some of which sound like they'd be more at home in television medical shows like ER, Grey's Anatomy, or House:

They apply lidocaine to induce a numbing sensation; 

Szechuan pepper to create a tingling feeling;

A cinnamon-derivative, cinnamaldehyde, stings;

The chili pepper derivative, capsaicin, warms the skin, and

Menthol provides a cooling sensation.

"By having the receptors in your skin generate the haptic sensations, we don't need bulky heating and cooling devices; we just need to deliver a few drops of a chemical stimulant, and your skin does the rest," says Lopes.

To test the idea, they developed a VR experience with a storyline about a failing nuclear reactor – and tested it on four Oculus Quest VR users wearing their chemical haptics kit, with a chemical reservoir behind the VR helmet. The narrative included a shorting-out reactor control panel, which produced sparks that excited the tingling sensation on the arm – and as people ran outside, the cooling chemical was applied to the face patch. The failure of a VR arm interface that controlled a door in the story was reinforced, emotionally, by numbing the user's arm with lidocaine, and a VR "wound" actuated the stinging stimulant. Finally, as a reactor door opened, the heat issuing from it was registered by the warming stimulant.  .... ' 

Glove Senses and Maps Tactile Stimuli

 Connecting more intelligently to the sense of touch.

Touchy-Feely Glove Senses, Maps Tactile Stimuli

By MIT News, August 9, 2021

A team of engineers at the Massachusetts Institute of Technology and China's Southern University of Science and Technology has designed a touch-sensing glove capable of "feeling" tactile stimuli.

An array of pressure-detecting sensors inside the glove can detect weak vibrations across the skin, such as from someone's pulse. The sensors operate similarly to humidity-measuring sensors but use human perspiration in place of a dielectric layer; two thin, flat electrodes on the skin form a circuit, and ions from moisture build on the underside of the sensing electrode in response to pressure, changing capacitance between both electrodes. The researchers enhanced the sensing electrode's sensitivity by lining it with thousands of gold bendable "micropillars."

The researchers believe the tactile glove could help to retrain motor function and coordination in people who have lost fine motor skills and could be modified to enhance virtual reality and gaming.

The team describes its work in "Skin-Electrode Iontronic Interface for Mechanosensing," published in Nature Communications.  

From MIT News  with images.

View Full Article   

Exploring Interactions with Haptic Feedback in Virtual Reality

Will this mean that people will be able to immerse themselves in games and imulation?   I am not much of a gamer, but like the idea that people will more realistic  'digital twins' to engage with physical objects and spaces.   Consider the future of that.  Its not only game-like controllers, but 'immersive interactions' that can enable us to be part of our physically enabled world.   Inside a 'digital twin'?  A powerful illusion indeed 

Microsoft Research collaborates with KAIST in Korea to explore bimanual interactions with haptic feedback in virtual reality

Published May 6, 2021

By Michel Pahud , Principal Research Software Development Engineer  Mike Sinclair , Senior Principal Researcher  Andrea Bianchi , Associate Professor at KAIST

Editor’s Note: Bimanual controllers are frequently used to enhance the realism and immersion of virtual reality experiences such as games and simulations. Researchers have typically relied on mechanical linkages between the controllers to recreate the sensation of holding different objects with both hands. However, those linkages cannot quickly adapt to simulate dynamic objects. They also make for bulky controllers that can’t be disconnected to support free, independent movements. This is the problem that researchers seek to solve in the recent paper titled “GamesBond: Bimanual Haptic Illusion of Physically Connected Objects for Immersive VR Using Grip Deformation”.

GamesBond is the outcome of a recent collaboration between Michel Pahud and Mike Sinclair from Microsoft Research and Andrea Bianchi, associate professor at KAIST and director of the MAKinteract lab, with two of his students, Neung Ryu, the original author of the paper, and Hye-Young Jo. The paper was accepted at ACM CHI Conference on Human Factors in Computing Systems (CHI 2021), where it received an honorable mention award.

In this project, we explored a pair of novel 4-DoF controllers, without actual physical linkage between them, that can bend, twist, and stretch together in concert to create the illusion of being connected as a single device with a physical link. Each controller can bend from 0 to 30 degrees in any direction, twist from -70 degrees to 70 degrees and stretch from -2.5mm to 9.0mm (the paper provides all the details of the mechanism).  ... " 

Haptics (Touch) Update from ACM

 A favorite topic, its part of how we understand reality, with good update here commissioned by ACM:

A Touch of Reality   By Samuel Greengard  Commissioned by CACM Staff

Haptics introduces ways to use virtual reality more effectively, to use robots for a wider array of tasks, and to produce biomechanical replacements for missing limbs that can replicate a sense of feel.

Imbuing robots and other machines with human-like sensory capabilities is an incredibly difficult task. In recent years, machine vision, hearing, and speech have advanced remarkably. Yet, the ability to building machines that can touch and feel the way people do has proven elusive.

All of this is about to change. Haptics—the use of technology to simulate touch, feeling, and motion—is finally hitting its stride. "A confluence of new actuator and sensor technology, along with ever increasing computing power, is pushing the technology into the mainstream," says Jake Rubin, founder and CEO of HaptX, a developer of haptics systems for robots and virtual reality (VR).

The technology introduces ways to use VR far more effectively—particularly for training in industrial settings. Yet haptics also makes it possible to use robots for a wider array of tasks, and to produce biomechanical limbs that replicate a sense of feel. "We are much closer to creating systems that talk to the brain or interact with nerves in ways that seems realistic," says Gregory A. Clark, director of the Center for Neural Interfaces at the University of Utah.

Out of the Body

Simulating human touch is an incredibly complex endeavor. It requires a deep understanding of physiology and neurobiology along with an ability to translate electrical signals into and out of binary code. Although VR, robotics, and biomechanical limbs incorporate and communicate touch in different ways, there's a common denominator, Clark says: a system ultimately must convince the brain that the signals flowing into it from machines are real.

HaptX, for example, has developed highly specialized gloves that use 130 tiny actuators and microfluidic air channels embedded in a fabric. The gloves displace the skin to 2 mm with no latency, producing ultrarealistic sensations; they also offer 6 degrees of freedom. "When you touch a virtual object, the pixel-like bubbles on this material inflate or deflate precisely to produce the same forces over the same area that that object would produce if it touched your hand in real life," Rubin explains.  ... ' 

Robotic Hands Closer to Human

New robotic hands with lots of possibilities for more dexterous manipulation 

Robot Hands One Step Closer to Human, Thanks to WMG AI Algorithms  By University of Warwick (U.K.)

Artificial intelligence algorithms enable the Shadow Robot Dexterous Hand to manipulate objects the same ways humans do.

Artificial intelligence (AI) algorithms developed by researchers at the Warwick Manufacturing Group (WMG) academic department of the U.K.'s University of Warwick enable the Shadow Robot Dexterous Hand to manipulate objects like humans do.

The Hand reproduces all of the degrees of freedom of a human hand.

The algorithms permit the Hand to learn how to coordinate movements and execute tasks such as throwing a ball and spinning a pen; the researchers said the algorithms can learn any task, as long as it can be simulated.

Said Warwick's Giovanni Montana, "The future of digitalization relies on AI algorithms that can learn autonomously, and to be able to develop algorithms that give Shadow Robot’s hand the ability to operate like a real one is without any human input is an exciting step forward.   ... " 

Mimicking the Sense of Touch

We examined how consumers interacted with product in retail.

Glove-Like Device Mimics Sense of Touch  University of New South Wales Sydney Newsroom,   Caroline Tang

A soft wearable device developed by engineers at the University of New South Wales (UNSW) Sydney in Australia recreates the sense of touch using haptic technology. The researchers created a three-way directional skin stretch device that is built into the fingertips of the haptic glove. Said UNSW's Thanh Nho Do, "Our soft, wearable haptic glove enables people to feel virtual or remote objects in a more realistic and immersive way. The inbuilt soft artificial muscles generate sufficient normal and shear forces to the user's fingertips via a soft tactor, enabling them to effectively reproduce the sense of touch."

Touch Sensors

Touch sensors are advancing.     We sought for understanding and adjusting how how products felt via a sensor.

OmniTact: A Multi-Directional High-Resolution Touch Sensor
Akhil Padmanabha and Frederik Ebert    May 14, 2020

Touch has been shown to be important for dexterous manipulation in robotics. Recently, the GelSight sensor has caught significant interest for learning-based robotics due to its low cost and rich signal. For example, GelSight sensors have been used for learning inserting USB cables (Li et al, 2014), rolling a die (Tian et al. 2019) or grasping objects (Calandra et al. 2017).

The reason why learning-based methods work well with GelSight sensors is that they output high-resolution tactile images from which a variety of features such as object geometry, surface texture, normal and shear forces can be estimated that often prove critical to robotic control. The tactile images can be fed into standard CNN-based computer vision pipelines allowing the use of a variety of different learning-based techniques: In Calandra et al. 2017 a grasp-success classifier is trained on GelSight data collected in self-supervised manner, in Tian et al. 2019 Visual Foresight, a video-prediction-based control algorithm is used to make a robot roll a die purely based on tactile images, and in Lambeta et al. 2020 a model-based RL algorithm is applied to in-hand manipulation using GelSight images.

Unfortunately applying GelSight sensors in practical real-world scenarios is still challenging due to its large size and the fact that it is only sensitive on one side. Here we introduce a new, more compact tactile sensor design based on GelSight that allows for omnidirectional sensing, i.e. making the sensor sensitive on all sides like a human finger, and show how this opens up new possibilities for sensorimotor learning. We demonstrate this by teaching a robot to pick up electrical plugs and insert them purely based on tactile feedback.  ... "

Artificial Skin and Augmented Reality

Links between skin and augmented reality of interest

Artificial Skin Could Help Rehabilitation, Enhance Virtual Reality
Ecole Polytechnique Fédérale de Lausanne
Laure-Anne Pessina

Researchers at Ecole Polytechnique Fédérale de Lausanne (EPFL) in Switzerland have developed a soft, flexible artificial skin, made of silicone and electrodes and equipped with soft sensors and actuators, which provides haptic feedback in the form of pressure and vibration. Strain sensors continuously measure the skin's deformation so that the haptic feedback can be adjusted to produce a realistic sense of touch. The actuators help form a membrane layer and can be tuned to varying pressures and frequencies. On top of the membrane layer sits a sensor that contains soft electrodes that measure the skin's deformation continuously and send the data to a microcontroller, which uses the feedback to fine-tune the sensation transmitted to the user. For now, the scientists have tested the technology on users' fingers. The next step, said EPFL's Harshal Sonar, "will be to develop a fully wearable prototype for applications in rehabilitation and virtual and augmented reality."

Simulating Softness

We were in a consumer space that had as one of it measures softness.   So the measurement and creation of softness was key.

How to simulate softness    by University of California - San Diego

UC San Diego researchers specially engineered a set of materials to mimic different levels of perceived softness. 

What factors affect how human touch perceives softness, like the feel of pressing your fingertip against a marshmallow, a piece of clay or a rubber ball? By exploring this question in detail, a team of engineers and psychologists at the University of California San Diego discovered clever tricks to design materials that replicate different levels of perceived softness.

The findings provide fundamental insights into designing tactile materials and haptic interfaces that can recreate realistic touch sensations, for applications such as electronic skin, prostheses and medical robotics. Researchers detail their findings in the Aug. 30 issue of Science Advances.

"We provide a formula to recreate a spectrum of softness. In doing so, we are helping close the gap in understanding what it takes to recreate some aspects of touch," said Charles Dhong, who co-led the study as a postdoctoral fellow at UC San Diego and is now an assistant professor in biomedical engineering at the University of Delaware. Dhong worked with Darren Lipomi, a professor of nanoengineering at UC San Diego and the study's co-corresponding author.

Based on the results from their experiments, the researchers created equations that can calculate how soft or hard a material will feel based on material thickness, Young's modulus (a measure of a material's stiffness), and micropatterned areas. The equations can also do the reverse and calculate, for example, how thick or micropatterned a material needs to be to feel a certain level of softness.  ....  "

Sensor Based Skin for Prosthetic Touch

New advances in touch.   We experimented with devices on the shelf to let consumers touch-engage with a product.    So this might be a way to quantitatively test the experience?

A sensor-filled “skin” could give prosthetic hands a better sense of touch

The “electronic skin,” inspired by the nervous system, can sense temperature, pressure, or humidity. It could be used to give prosthetic limbs a more complex sense of touch.

Humans are amazing: Your body is a sensing machine, thanks to the roughly 45 miles of nerves inside your body that connect your skin, brain, and muscles. A team from the University of Singapore has now used that nervous system as inspiration to create a "skin" for robots that, one day, could improve their ability to detect and understand their environment.

How it works: Sheets of silicon were covered with 240 sensors that can pick up contact, pressure, 
temperature, and humidity. These are able to simultaneously transmit all this data to a single decoder, and should still work when the system is scaled up to 10,000 sensors, according to Benjamin Tee, the coauthor of the study, which was published in Science Robotics today.

What’s new: Flexible robotic “skin” has been tested in previous studies, but this system is the first to enable many sensors to feed back to a single receiver, allowing it to act as a whole system rather than a bunch of individual electrodes, Tee said. Crucially, it still works even if the individual receptors are damaged, making it more resilient than previous iterations.  .... " 

The Sleeve Communicates Emotions through Touch

Sounds remarkable,  good article.  Had a long time look at how people reacted emotionally to product.

Gadget Communicates Human Emotions through Touch , by Elizabeth Stinson on Wired.

ASK ENGINEERS WHAT the future of communication looks like and they’ll show you a fiber-optic cable. Ask artists and they’ll conjure something like the Sleeve. For the past year, engineers at Nokia Bell Labs, the famed New Jersey research facility that birthed the transistor, have been developing this wearable armband with input from artistic collaborators. “We’re reductionist in our thinking; artists are divergent,” says Domhnaill Hernon. He's the head of a ­program called Experiments in Art and Technology, founded in the ’60s and newly resurrected in partnership with the design incubator New Inc. In this right-brains-meet-left coalition, engineers and artists team up to explore big questions: Can humans communicate through touch? Is it possible to transfer empathy? What’s the successor to smartphones? The Sleeve tries to answer them. This early model gathers information about the user’s physical and emotional state through gyroscopes, accelerometers, and optical sensors, then communicates that intel via haptic pulses and screen-displayed messages. The collaborators aim to inspire more engineers to consider the emotional plane. Soon you’ll be able to express your heart through your sleeve.  .... "

Graphene Feeling Sensor

Another example of connecting a new kind of sensor to gather new data, which can then be connected to machine learning methods.    The value of a sensor is then if it can produce data that is useful for training.

Graphene-Based Sensor Learns to Feel Like a Human 
Chemistry World  By Hannah Kerr

Researchers at Hanyang University in South Korea have integrated an electric sensor with a machine learning program, creating a device that can differentiate between surface textures, with potential applications in virtual reality, robotics, and medical prosthetics. The sensor is fabricated from a graphene-flake film deposited onto a polyethylene naphthalate substrate. The device registers changes in electrical conductance and resistance via the film when strain causes deformation, boosting the physical contact between individual flakes in the film. The machine-learning program applies the sensor's conductance and resistance data to define specific features connected with different surface texture types. The researchers have applied the graphene film to an artificial fingerprint structure so it reacts to tiny vibrations caused by the ridges on the fingerprint rubbing against a textured surface; the sensor analyzes these signals to identify the "feel" of differently textured fabrics. In a blind test of 50 people, the sensor outperformed humans in classifying 12 new fabrics. ... "

Whiskers as Sensors

Tactile sensors, like the idea of using new kinds of sensors.  Imagine the increased number of data points that could be gathered and fed to deep learning techniques.   Added to visual sensory input.

Want a Robot that can really feel?  Give it Whiskers.   By Matt Simon in Wired
" ... Whiskers are all the rage in nature, so why not give them to robots? Mechanical engineer Mitra Hartmann of Northwestern University is doing just that. In a new paper published in the journal Soft Robotics, Hartmann and her team detail how they’ve pulled one step closer to a rat-like machine that can feel an object and pinpoint it in 3-D space. Meaning robots of all kinds could soon get a powerful new sense. ... " 

Artificial Fingertip Feels

Artificial fingertip that ‘feels’ wins Harvard's robotics competition

TacTip, a 3D-printed tactile sensor  (University of Bristol) 

An open-source 3D-printed fingertip that can ‘feel’ in a similar way to the human sense of touch has won Harvard University's international Soft Robotics competition for its contribution to soft robotics research.  ....   "