Spray on Smart Skin

Examined a number of related methods, without success. 

Spray-on Smart Skin Reads Typing and Hand Gestures No cameras, keyboards, sensor gloves, or VR motion trackers required      PRACHI PATEL  Spectrum IEEE

KYUN KYU “RICHARD” KIM/  STANFORD UNIVERSITY

A new AI learning scheme combined with a spray-on smart skin can decipher the movements of human hands to recognize typing, sign language, and even the shape of simple familiar objects. The technology quickly recognizes and interprets hand motion with limited data and minimal training and should work for all users, its developers say.

Besides finding use in gaming and virtual reality, the new hand-task-cognition technology could allow people to communicate with others and with machines using gestures. Other applications the technologists envision include surgeons remotely controlling medical devices, as well as a new modality for robots and prosthetics to achieve object and motion recognition.

The gesture-recognition technology developed so far has relied on bulky wrist bands that measure electrical signals produced by muscles or on wearable gloves with strain sensors at each joint. Other approaches have involved cameras that track human motion and interpret it using machine learning. Those motion-capturing camera systems require images taken from multiple surrounding angles, which means multiple cameras are needed for a single gesture-recognition system. These multicam systems also suffer from the inherent limitations of vision-based sensors, says Sungho Jo, a professor in the school of computing at the Korea Advanced Institute of Science and Technology. Such limitations include regions of a workspace that are not covered by multiple cameras, as well as errors that inevitably occur when a hand or other object is occluded from view.

The software used so far has also been cumbersome. Researchers have typically relied on machine-learning models based on supervised learning algorithms that are computationally intense. They require that a large amount of data be collected for each new user and task, all of which require humans to label.  ... ' 

Eyelid Geatures for Motor Impairment

Always thought there would be value in the eyelid asa secondary communication at least.  This is a considerable and technical article.

Eyelid Gestures for People with Motor Impairments

By Mingming Fan, Zhen Li, Franklin Mingzhe Li

Communications of the ACM, January 2022, Vol. 65 No. 1, Pages 108-115

Although eye-based interactions can be beneficial for people with motor impairments, they often rely on clunky or specialized equipment (e.g., stationary eye-trackers) and focus primarily on gaze and blinks. However, two eyelids can open and close in different orders and for different duration to form rich eyelid gestures. We take a first step to design, detect, and evaluate a set of eyelid gestures for people with motor impairments on mobile devices. We present an algorithm to detect nine eyelid gestures on smartphones in real time and evaluate it with 12 able-bodied people and 4 people with severe motor impairments in two studies. The results of the study with people with motor-impairments show that the algorithm can detect the gestures with .76 and .69 overall accuracy in user-dependent and user-independent evaluations. Furthermore, we design and evaluate a gesture mapping scheme for people with motor impairments to navigate mobile applications only using eyelid gestures. Finally, we discuss considerations for designing and using eyelid gestures for people with motor impairments. ..... '

Algorithm Improves Hand Gesture Complexity

Hoe much have generalized gestures use?  Considerations? 

Algorithm Improves Hand Gestures   By SPIE, December 30, 2021

A team led by Zhiyi Yu of Sun Yat-sen University recently developed a new hand gesture recognition algorithm that strikes a balance between complexity, accuracy, and applicability. The team adopted innovative strategies to overcome key challenges and realize an algorithm that can be easily applied in consumer-level devices, as detailed in their paper published in the Journal of Electronic Imaging.

A main feature of the algorithm is adaptability to different hand types. "By first classifying the input gesture by hand type and then using sample libraries that match this type, we can improve the overall recognition rate with almost negligible resource consumption," Yu says.

Another key aspect is the use of a "shortcut feature" to perform a prerecognition step. "The gesture prerecognition step not only reduces the number of calculations and hardware resources required but also improves recognition speed without compromising accuracy," Yu says.

Tests showed that the approach could recognize hand gestures in real time with an accuracy exceeding 93%.

From SPIE Full Article