AI Model Digests Video to Learn Sign Language

AI Model Digests Video to Learn Sign Language

New Atlas

Paul McClure, May 24, 2023

A tool developed by researchers at Spain’s Barcelona Supercomputing Center and the Universitat Politècnica de Catalunya (UPC) uses artificial intelligence (AI) to improve sign language translation. The researchers fed a transformer-style machine learning model more than 80 hours of video in American Sign Language with corresponding English transcripts from the How2Sign dataset. The researchers addressed the variability and complexity of sign languages by pre-processing the video data to extract spatiotemporal information via the Inflated 3D Networks technique. They found the model produced meaningful translations, although they agreed, "There is still room for improvement."

Microsoft Teams Gets Improved Sign Language

 New accessibility

Microsoft Teams is getting a sign language view  in theVerge

The new accessibility feature creates a larger, high-quality view for interpreters and signers separate from other call participants or presentations.

By JESS WEATHERBED   Nov 18, 2022, 7:57 AM EST|

A screenshot of Microsoft Teams new sign language view. A signer is prioritized on the left, with another call participants kept separately on the right.

Sign language view allows signers to prioritize up to two video streams on center stage, in a consistent location during every meeting. Image: Microsoft

Microsoft has a new dedicated sign language view for its Microsoft Teams video conferencing platform. The feature enables sign language users to select up to two other participants’ videos to be fixed in a central location, allowing designated signers to be visible throughout the meeting.

Microsoft says Teams users can enable the feature for all meetings or on a meeting-by-meeting basis, which is especially useful for non-hearing impaired signers. Sign language interpreters can be pre-assigned prior to starting a meeting if they’re within your organization, and external interpreters can only be assigned within a meeting using the Accessibility tab in the app settings. Microsoft does not provide sign language interpreters for Teams meetings.

Sign language view doesn’t affect what other call participants can see on Teams

Other participants can still be pinned without encroaching on the prioritized video streams of the signer, which will default to an uncropped ratio at the highest available quality. The prioritized stream shifts when a presentation is shared but will still be easily distinguishable from other participants, retaining a larger size and high video quality. Sign language view is also personal to the user — other call participants will not be alerted or have their own Teams view altered.

Microsoft is also announcing a new sticky Accessibility preferences pane for Teams. The new pane gives users easy access during calls to make changes like turning on sign language view by default or pre-identifying preferred signers. Sign language view and live captions can also be quickly toggled on or off within the pane. ...'

Humanoid Sign Language

Robotic Sign Language

A robotic hand making a gesture in sign language. UC3M Programs a Humanoid Robot to Communicate in Sign Language 
Carlos III University of Madrid (Spain)
July 8, 2019

The_UC3M_programmes_a_humanoid_robot_to_communicate_in_sign_language
Researchers at Carlos III University of Madrid (UC3M) in Spain have programmed a humanoid robot, named TEO, to communicate in sign language. The team used simulations to indicate the specific position of each phalanx (finger bone) in depicting specific signs from Spanish Sign Language. The researchers then reproduced each position with the robot’s hand, trying to make the movements similar to those a human hand would make. To date, TEO has mastered the fingerspelling alphabet of sign language, as well as a very basic vocabulary related to household tasks. Said UC3M researcher Jennifer J. Gago, "The deaf people who have been in contact with the robot have reported 80% satisfaction, so the response has been very positive."

Echo Mod for Sign Language

When I saw this, appeared that this is a hard thing to do, and apparently the range of signs and capabilities are still limited, but a remarkable possibility.  For Echo devices that include cameras, the system detects and interprets sign language.

This Amazon Echo mod lets Alexa understand sign language
‘If voice is the future of computing, what about those who cannot speak or hear?’   By James Vincent   @jjvincent  in theverge