Eyelid Geatures for Motor Impairment

Always thought there would be value in the eyelid asa secondary communication at least.  This is a considerable and technical article.

Eyelid Gestures for People with Motor Impairments

By Mingming Fan, Zhen Li, Franklin Mingzhe Li

Communications of the ACM, January 2022, Vol. 65 No. 1, Pages 108-115

Although eye-based interactions can be beneficial for people with motor impairments, they often rely on clunky or specialized equipment (e.g., stationary eye-trackers) and focus primarily on gaze and blinks. However, two eyelids can open and close in different orders and for different duration to form rich eyelid gestures. We take a first step to design, detect, and evaluate a set of eyelid gestures for people with motor impairments on mobile devices. We present an algorithm to detect nine eyelid gestures on smartphones in real time and evaluate it with 12 able-bodied people and 4 people with severe motor impairments in two studies. The results of the study with people with motor-impairments show that the algorithm can detect the gestures with .76 and .69 overall accuracy in user-dependent and user-independent evaluations. Furthermore, we design and evaluate a gesture mapping scheme for people with motor impairments to navigate mobile applications only using eyelid gestures. Finally, we discuss considerations for designing and using eyelid gestures for people with motor impairments. ..... '

Eye Movements Predicting Mental Health

What appears to be a reasonable pattern detection type application

AI gauges the mental health of cancer patients through eye movements

It's an early project, but shows potential.

Jon Fingas, @jonfingas  in Engadget

Good mental health is important, and early research suggests AI might help diagnose that health for people who are under a particularly heavy strain. Scientists have developed a combination of deep learning algorithms that use eye tracking to gauge the mental health of cancer patients after surgery. Ideally, this would help spot patients likely to be suffering from anxiety or depression when a human can’t perform an initial psychological assessment.

The system uses a mix of a convolutional neural network and long short-term memory algorithms to study the eye movements of patients wearing tracking glasses (in this case, Tobii Pro 2 glasses) while they contemplate artwork. The AI used the gaze and pupil position data from those glasses to determine how likely someone was to raise concerns on established hope, anxiety and mental wellbeing questionnaires they would fill out later.  ... ' 

Flight Simulator Eye Tracking

More advances in eye tracking and associated  uses:

Tracking the Eye of the Pilot
ETH Zurich
Michael Keller
November 25, 2019

Researchers at the Swiss Federal Institute of Technology (ETH) Zurich, Swiss International Air Lines, the U.S. National Aeronautics and Space Administration, and others have developed and tested eye-tracking software to train pilots. The Instructor Assistant System (iAssyst) software lets instructors analyze the gaze patterns of trainees in the cockpit of a flight simulator to see how they monitor the automated systems of modern passenger planes. iAssyst combines video, audio, and simulator recordings, while displaying pilots' gaze patterns. iAssyst permitted trainers to analyze pilots' flying performance with greater precision, enabling instructors to better evaluate the causes of potential pilot errors, and appropriately adjust the training regimen. ETH researchers Martin Raubal and David Rudi said the software also could be used for medical training. ... " 

Robotic Lenses

In New Scientist, complete article requires subscription:

A robotic lens can be controlled by simply looking around or blinking   By Leah Crane

Blink twice to zoom in. A new soft lens can be controlled by your eye movements, pivoting left and right as you look around and zooming in and out when you blink.

The human eyeball is electric – there is a steady electrical potential between its front and back, even when your eyes are closed or in total darkness. When you move your eyes to look around or blink, the motion of the electrical potential can be measured. Shengqiang Cai at the University of California San Diego and his colleagues used these signals, …

Hololens 2 to have Eye Movement Detection

Good addition of detecting and I assume controlling with eye movement.   The control by gesture has never been good enough for many applications.  Need more general applications to get people used to the interaction

Microsoft HoloLens 2 augmented reality headset unveiled  By Leo Kelion in BBC Tech.

Microsoft has a new version of its augmented reality headset, which now detects where its users are looking and tracks the movements of their hands.

It said that HoloLens 2 wearers would find it easier to touch and otherwise interact with graphics superimposed over their real-world views.

Other improvements include filling more of a user's view and automatically recognising who they are.

The firm is pitching the kit as being ready for use in business environments.

Many experts believe mixing together graphics and real-world views has greater potential than virtual reality, which removes the user from their immediate environment. .... "

Eye Tracking Important

Good to see the mention of Zaltman's work, whose expertise we used extensively.    Not quite sure  what 95% means here, but certainly important.

Why is Eye-Tracking really important for Market Research?  in DSC

Posted by Ayush Srivastava in DSC

“95% of the purchase decisions happen in the subconscious”

Yes, you read it right! 95% of the purchase decisions happen in our sub-conscious — according to research performed by Harvard Business School professor Gerald Zaltman. For the uninitiated, or someone new to the consumer insights industry, this is a revelation. Indeed, when I think of it, I cannot give a reasonable explanation to why I chose to drink orange juice over apple juice at a party last night. Both the options were in front of me, some neurons fired in my brain and my hand just steered towards the orange juice.

This research from Prof. Gerald also made me think that how companies can understand what’s happening in consumers subconscious in order to derive insights. And it is important to figure this out because asking consumers (surveys, interviews etc) can only yield ‘conscious’ reasons behind their actions, which explains only 5% of the decisions as per Prof. Gerald. People usually decide in their subconscious and then give a logical explanation according to their “conscious brain” ... "

Eye Tracking Helps Augmented Reality

Pointed at by some of our retail experiments.

Eye-Tracking Technology Could Help AR 
The Dartmouth    by Anthony Robles

Dartmouth College researchers have developed battery-free eye-tracking glasses that could enhance augmented reality (AR) technologies. Dartmouth's Tianxing Li and Xia Zhou designed the eye tracker to fit into ordinary glasses and identify the location and size of the pupil. Zhou said the solution can optimize rendering for AR glasses, with potential medical applications that include "health monitoring" and evaluating "the effectiveness of certain clinical treatments." AR gamers also could benefit, since the technology allows them to use eye movements as an "interface to interact with the virtual scene." According to Zhou, eliminating the battery pack makes it possible to "continuously track eye movements" without requiring a recharge. Added Li, “One of the goals for the project is that we want to push human sensing functionality to its limit.”  ... ' 

AI Empowered Coding for Eye Tracking

Brought to my attention.  A natural next step

Smart-Gaze: AI powered coding for efficient Eye Tracking   in Karna.ai
“What you see is what you buy”

It is believed that 95% of the purchase decisions happen in the subconscious (let’s call it the reptile brain). The decisions taken by the reptile brain are strongly influenced by what we see. When walking across a supermarket aisle, we see hundreds of packs of different products. But then, there are some products that capture our attention. When we see such products, the reptile brain kicks into action, it fixates its attention on that packaging that looks interesting (“ooh shiny!!!”) and before we know it — we enter the process of seriously considering to purchase that product.

Eye-tracking is a well known tool to implicitly measure how people respond to different product packaging, advertisement copies, web-page layouts, banner placements and more. Performing eye-tracking studies have the potential to generate tremendous RoI on product packaging, advertisement creatives and placement decisions. But, we (and the clients we speak to) believe that the industry is only beginning to scratch the surface when it comes to using this technique for generating insights. The reasons cited include that it is — expensive, slow, not objective (as participants get influenced by the process). However, we believe that the key reason is that the technology is not that developed yet. 
... " 

AI Solutions for Market Research. Reach us at http://karna.ai/

Eye Tracking API in Windows

Newly announced. Broader accessibility and data tracking capabilities.

How to create accessible apps and immersive game experiences with new eye tracking APIs   By Jacob Cohen / Program Manager, Windows Input Platform Team

Last year in the Windows 10 Fall Creators Update, we introduced Eye Control, an experience that allows customers to control Windows using only their eyes. We continue to design with individuals like Steve Gleason in mind, and always challenge ourselves to build products that create positive change. We understand, however, that fulfilling our mission statement – “to empower every person and organization on the planet to achieve more” – goes beyond the product alone. It starts with something more foundational: building a platform to empower all developers to create products and experiences that can help improve people’s lives and make a positive impact.  .... " 

VR and Eye Tracking

We saw the need for this early on.   Where you are looking implies intent, so why not track it?

Peter Rubin in Wired.
Eye Tracking Coming to VR Sooner that you think.   What Now?  .... 

".... Eye-tracking has been a part of the VR conversation for years; a company called FOVE even crowdsourced its own eye-tracking headset in 2015. Yet, when the feature would actually arrive at scale has been an open-ended question. Even as multiple companies race to release all-in-one "standalone" headsets—Lenovo's Mirage Solo launches in May, with Oculus' $199 Go device rumored to be close behind, and HTC's Vive Focus will be arriving stateside this year after a China-only debut—the dream of our eyes making it into VR seemed farsighted at best. ..."

Critical View of Data Viz Eye Tracking

Data viz expert Stephen Few looks at work done by Tableau using eye tracking methods to get insight about designing data visualizations.    I recall seeing some previous research in this area.   It is not too unlike similar work being done tracking how people look at advertising or placement on store shelves.   Where do people look, what do they see, what do they retain,  is what we think is important even noticed?   Stephen Few points out the issues.

Tracking Ad Effects Offline

Snap Seeking offline ad success measures:   In Adage: 

" ... Snap Inc. has acquired Placed, a startup that measures the offline success of digital advertising campaigns, for about $125 million, according to people familiar with the matter.

Including stock payouts, the full value of the deal could exceed $200 million, said two other people with knowledge of the deal, who asked not to be named because the terms aren't being disclosed. Snap confirmed the acquisition on Monday, while declining to comment on the price.

The acquisition will help Snap expand its efforts to show that ads on its photo and video-sharing app Snapchat are driving users to stores. Placed, based in Seattle, will continue to operate independently, with David Shim, the startup's chief executive officer, reporting to Snap's chief strategy officer, Imran Khan. .... " 

Eye Tracking and Virtual Reality

More developments in the use of biometric capabilities to better understand human reactions.  Its natural to think about how VR methods technology could be linked to these.  This then points to better measurement of cognitive reactions.   The reactions are to virtual environments, which can be readily changed to test hypotheses.

Sensor Tracks Eye Movements in Real Time to Enhance Virtual Reality

The Engineer (United Kingdom)    by Helen Knight

A team of researchers from Belgium and the Netherlands has developed a sensor that can detect eye movements in real time using an array of electrodes. The researchers say the technology could enable videogamers to interact with the screen by controlling cursors with their eyes to navigate through menus and select different options, or to open and close applications. In addition, the technology could provide feedback to the game on how the user is reacting to their virtual surroundings, according to Carlos Agell at the Belgium-based IMEC innovation center. The team developed a system based on four electrodes, which are built into the glasses around each lens, at points where the frames touch the skin in order to collect electrical impulses emitted by the eye. Agell says the algorithm translates the signals into a position, based on the angle the eye is making with its central point of vision. ... " 

IMotions

Was brought back to my attention.  Biometrics typically need multiple dimensions and laboratory style data handling environments.

Scalable Biometric Research Solutions
Since 2005 iMotions has developed our software platform that lets researchers integrate best-in-class biosensors, eye tracking, facial expression analysis, EEG, GSR, EMG, ECG and Surveys in one unified software platform. .... " 

Oculus and Eye Tracking

A natural next step, for both games and more serious applications.  In Engadget:

Oculus now owns an eye-tracking company
The Eye Tribe's visual tracking technology could be in the next Oculus Rift.   by Sean Buckley  .... 

Controlling Smartphones With Your Eyes

In CACM:
" ... In an effort to make eye tracking cheap, compact, and accurate enough to be included in smartphones, a group of researchers is crowdsourcing the collection of gaze information and using it to teach mobile software how to figure out where you’re looking in real time.  ... " 

Gazerecorder Eye Tracking

Brought to my attention by a client connection.    In Sourceforge, a new version 1.9 of Gazerecorder  " ... WebCam Eye Tracking for usability testing ... " .  See also their web site.    A broad overview with images. 

Automating Multiple Streams of Biometric Analysis

I see that iMotions is out with a method that can analytically combine eye tracking, facial expression, ECG and more.   Have not used their devices and methods, but combining streams of data would be interesting.  This is analogous to ensemble methods in analytics.  Does this further need a cognitive, in a true sense, method to combine results?    More here.

Eye Tracking Study of Product Similarity on Brand Location and Recognition.

Of retail interest from a recent conference:  " ... Para-Sight: A visual search study into the effects of product similarity on brand location and recognition.   by Tim Holmes  ... ".    Good to see this kind experimental research being done with eye tracking.  Now how to effectively apply it.  

Recognizing Gaze Direction with Machine Learning

In Technology Review:

Virtual Eyes Train Deep Learning Algorithm to Recognize Gaze Direction

Gaze estimation is a classic problem of machine vision, which can now be solved by one computer training another.

Eye contact is one of the most powerful forms of nonverbal communication. If avatars and robots are ever to exploit it, computer scientists will need to better monitor, understand, and reproduce this behavior.

But eye tracking is easier said than done. Perhaps the most promising approach is to train a machine-learning algorithm to recognize gaze direction by studying a large database of images of eyes in which the gaze direction is already known.  ... "