Microsoft and PeopleLens

Fascinating direction, and natural for Msoft's work in AR capabilities. 

Microsoft Research Blog

PeopleLens: Using AI to support social interaction between children who are blind and their peers

Published March 14, 2022

By Cecily Morrison , Principal Research Manager  Katherine Jones , PhD student  Martin Grayson , Principal Research Software Development Engineer  Ed Cutrell , Sr. Principal Research Manager

The PeopleLens is a new research technology designed to help people who are blind or have low vision better understand their immediate social environments by locating and identifying people in the space. Coupled with a scheme of work based on research and practices from psychology and speech and language therapy, the system can help children and young people who are blind more easily forge social connections with their peers.

For children born blind, social interaction can be particularly challenging. A child may have difficulty aiming their voice at the person they’re talking to and put their head on their desk instead. Linguistically advanced young people may struggle with maintaining a topic of conversation, talking only about something of interest to them. Most noticeably, many children and young people who are blind struggle with engaging and befriending those in their age group despite a strong desire to do so. This is often deeply frustrating for the child or young person and can be equally so for their support network of family members and teachers who want to help them forge these important connections.

PUBLICATION

PeopleLens 

The PeopleLens is a new research technology that we’ve created to help young people who are blind (referred to as learners in our work) and their peers interact more easily. A head-worn device, the PeopleLens reads aloud in spatialized audio the names of known individuals when the learner looks at them. That means the sound comes from the direction of the person, assisting the learner in understanding both the relative position and distance of their peers. The PeopleLens helps learners build a People Map, a mental map of those around them needed to effectively signal communicative intent. The technology, in turn, indicates to the learner’s peers when the peers have been “seen” and can interact—a replacement for the eye contact that usually initiates interaction between people.   ... ' 

Smart Shoe for the Blind

Perhaps unusual means of protection for the sight impaired.

 'Intelligent' Shoe Helps Blind People Avoid Obstacles

The Daily Mail (U.K.), Jonathan Chadwick, May 5, 2021

Computer scientists at Austria's Graz University of Technology (TU Graz) and medical technology maker Tec-Innovation have designed a shoe with ultrasonic sensors that can detect obstacles and emit vibrations to warn blind and visually impaired wearers. The sensors are attached to the toe of each InnoMake shoe, which will sell for £2,700 ($3,759) per pair, and the vibrations become faster the closer the wearer gets to an obstacle. Tec-Innovation's Markus Raffer said the sensors detect obstacles up to four meters (13 feet) off, with the device able to identify the nature of an obstacle and its directional path, especially if it is downward-facing.

Microsoft says its AI can describe images 'as well as people do'

 Recall some portion of this claim being made, but had not heard much new from Microsoft,  Know sight impaired people who could use the capability.     We also worked on the general idea of  'captioning', which turns out to be tough to do generally well.  

Microsoft says its AI can Describe Images 'as Well as People Do'  By Devindra Hardawar, @devindra  in Engdget 

It’s a new milestone for AI that could genuinely help the visually impaired. 

Describing an image accurately, and not just like a clueless robot, has long been the goal of AI. In 2016, Google said its artificial intelligence could caption images almost as well as humans, with 94 percent accuracy. Now Microsoft says it’s gone even further: Its researchers have built an AI system that’s even more accurate than humans — so much so that it now sits at the top of the leaderboard for the nocaps image captioning benchmark. Microsoft claims its two times better than the image captioning model it’s been using since 2015. 

And while that’s a notable milestone on its own, Microsoft isn’t just keeping this tech to itself. It’s now offering the new captioning model as part of Azure's Cognitive Services, so any developer can bring it into their apps. It’s also available today in Seeing AI, Microsoft's app for blind and visually impaired users that can narrative the world around them. And later this year, the captioning model will also improve your presentations in PowerPoint for the web, Windows and Mac. It’ll also pop up in Word and Outlook on desktop platforms.

Smart Windows from Ford

Advanced accessibility play of interest.  In Engadget.

Ford’s smart windows can help blind passengers take in the view
It converts the view into vibrations that they can feel on the window. .... " 

by Mallory Locklear, @mallorylocklear