Speech AI Spotlight

 Would expand potential capabilities for AR ... makes me broaden they potential usefulness.

Speech AI Spotlight: Visualizing Spoken Language and Sounds on AR Glasses

Jun 23, 2023

By Sirisha Rella

Audio can include a wide range of sounds, from human speech to non-speech sounds like barking dogs and sirens. When designing accessible applications for people with hearing difficulties, the application should be able to recognize sounds and understand speech.

Such technology would help deaf or hard-of-hearing individuals with visualizing speech, like human conversations and non-speech sounds. Combining speech and sound AI together, you can overlay the visualizations onto AR glasses, making it possible for users to see and interpret sounds that they wouldn’t be able to hear otherwise. 

According to the World Health Organization, about 1.5B people (nearly 20% of the global population) live with hearing loss. This number could rise to 2.5B by 2050.

Cochl, an NVIDIA partner based in San Jose, is a deep-tech startup that uses sound AI technology to understand any type of audio. They are also a member of the NVIDIA Inception Program, which helps startups build their solutions faster by providing access to cutting-edge technology and NVIDIA experts.

The platform can recognize 37 environmental sounds, and the company went one step further by adding cutting-edge speech-to-text technology. This gives a truly complete understanding of the world of sound.

AR glasses to visualize any sound

AR glasses have the potential to greatly improve the lives of people with hearing loss as an accessible tool to visualize sounds. This technology can help enhance their communication abilities and make it easier for them to navigate and participate in the world around them.  ... ' 

Glasses Sized VR From Stanford and Nvidia

Making VR Headset Glasses-Sized.  Is this good enough for a typical user?

ACM TECHNEWS

VR Headsets Reduced to Size of Regular Glasses

By Interesting Engineering,  August 15, 2022

Virtual reality (VR) headgear has been reduced to the thickness of a pair of regular glasses by scientists at Stanford University and software company Nvidia, who engineered pancake lenses to interoperate with three-dimensional images. Researchers also used a phase-only spatial light modulator to shorten the distance between the lens and the display, creating “a small image behind the device."

The device weighs just two ounces, much less than Meta's Quest headset, which weighs roughly a pound. However, the prototype's field-of-view is much smaller than that of commercially available headsets, and it requires accurate measurement of the user's pupil.

From Interesting Engineering  

Euro Glasses

 Had not seen this as yet, note these are developed with Chinese manufacturer Nreal. 

AR Glasses Allow Deaf People to 'See' Conversations by Turning Audio into Subtitles

EuroNews

Aisling Ní Chúláin, July 29, 2022

U.K. startup XRAI Glass has unveiled new augmented reality (AR) glasses that can translate audio into subtitles for deaf and hard-of-hearing users. The XRAI Glasses, developed with Chinese AR glasses manufacturer Nreal, are connected to a mobile phone that manages processing and graphics generation. "What our software effectively does is it takes an audio feed from the microphone on these glasses [and] sends it down to the phone," said XRAI Glass' Dan Scarfe. "On the phone we are effectively turning that audio into closed captions and then, using that Nreal software, we can project those subtitles into the real world."

Full Article

NReal Glasses

 Still awaiting these,

NReal AR glasses set to launch this spring in UK

Details on a release date of the product’s release or pricing remain unclear

Chinese augmented reality (AR) startup Nreal will be bringing its smart glasses to the U.K. this spring. 

British mobile operator EE, which is owned by telecoms group BT, confirmed the news Tuesday, saying it has partnered with the Beijing-based company to "exclusively bring the Nreal Air AR glasses to the UK’s best network for 5G." ... ' 

Metaverse Wearable Device

A Meta-Facebook device on every face?

Metaverse wearable devices ‘could be as big as phones,’ Qualcomm CEO says  in Yahoo Finance

by Julie Hyman·Anchor

Wed, November 17, 2021, 2:57 PM·2 min read

The company whose chips power Facebook’s Oculus virtual-reality headset has already tapped into the metaverse opportunity. Now Qualcomm’s CEO says glasses could eventually be as widespread as smartphones. 

Qualcomm CEO Cristiano Amon knows smartphones well. The company’s main business is providing semiconductors for those devices. As Apple turns toward manufacturing its own chips for its phones, Qualcomm has been working on diversifying its revenue into products and services for the automotive industry, the Internet of Things — and the metaverse. They’re not new entrants into the market. 

“For all of the devices that are being commercially deployed, you have one thing in all of them — which is Snapdragon XR,” Amon told Yahoo Finance Live, referencing Qualcomm’s extended reality version of its main Snapdragon platform. “So we’re very excited about that opportunity, and it could be as big as phones, if you have companion glasses that you carry together with your smartphone.” 

Amon pointed to Facebook’s name change as one marker of success for that company’s XR device: "Even looking at how the company is now called Meta is a reflection of how successful Oculus has been,” he said.

Metaverse products were one aspect of the diversification strategy Amon laid out during Qualcomm’s investor day on Tuesday. He outlined how Qualcomm is working on reducing its reliance on Apple, and is targeting $46 billion in revenue by 2024, including $9 billion from the Internet of Things. ... ' 

More Smart Glasses

Additional experiments with the idea.  Will this become common, replacing phones and watches?

TCL unveils Thunderbird Smart Glasses with a full-color transparent micro-LED display

PETER, 15 OCTOBER 2021

There is a renewed push to make smart glasses the next big thing and these may be the most impressive yet – TCL unveiled the Thunderbird Smart Glasses Pioneer Edition. They use a micro-LED display with a wave guide that was developed in-house over the last three years. It’s a transparent color display, so these look and feel like regular glasses. 

This sets them apart from Xiaomi’s concept smart glasses, which only featured a monochrome display also using micro-LED. In both cases the LEDs are tiny, just 4 µm in size, but TCL’s team developed a proprietary algorithm that enables the Thunderbird glasses to show a full-color image.

Check out the video below for a demonstration of what one can do with smart glasses. Basically, the aim is to replace your smartphone or at least its display to start. You can read the news, view messages, check your schedule, control your smart home and so on, all with the added advantage that the glasses are a display that’s always in front of you.  ...' 

Updates for Apple Glass (Updated with a number of competitors)

Following Apple's look at Smart Glasses.   Should note that Google Glasses are apparently still being used in test in some specialized situations.  But not in any standard way to interact with the real world.  One of the primary issues has been the consideration of privacy.   Is someone wearing the glasses using a front facing camera to record me?     Is there a good way to alert me of that?  Like showing they are turned off, and is that trustable?   I had noted that it had been rumored that the Apple version would not have any forward facing camera at all.  But only a Lidar scanner. Below some recent information. 'Every thing they know ...  But note some of this is clearly speculation.  We did early work in manufacturing maintenance using the idea.   I have a potential client in the space.  

See below and also from TechValley:  https://www.youtube.com/watch?v=JLNvSYr4eeI   Which shows some good, but also speculative consumer uses.

UPDATE:   Pointed out to me:   7 Best Smart Glasses 2021 Take Your Life Far Away:  https://www.youtube.com/watch?v=UbWiCzpnCTY  Video which also includes some useful novel uses. 

Apple Glasses: Release date, price, features and leaks,  Excerpt from Tom'sGuide,  By Kate Kozuch 17 days ago

Apple Glass could redefine wearable computing. Here's everything we know so far.

The Apple Glass augmented reality glasses could be a futuristic wearable product that overlays graphics in the real world around you.

When we first heard word of "Apple Glasses," rumors suggested the lenses would launch as soon as this year. But based on newer leaks and insider intel, the near future Apple augmented reality glasses looks blurry.

A longtime source for Apple product releases believes the release won't happen until 2022. Bloomberg's Mark Gurman said the earliest we'll see them is 2023. Either way, the project is definitely in the works.

Apple Glass is expected to run on Starboard (or perhaps glassOS,) a proprietary operating system uncovered in the final version of iOS 13. The augmented reality framework shows up multiple times in code and text documents, meaning Apple is likely testing activation and application. 

Here's everything else we know about Apple Glass, including the potential release date, price, design and specs.   ....' 

Italian Data Protection Authority Queries Smart Glasses

 I thought there might be some regulatory push back on Facebook smart glasses that gather and store data and images, here perhaps the first case.    

Italy Data Authority Asks Facebook for Clarifications on Smart Glasses

Reuters, Elvira Pollina, September 10, 2021

Italian data protection authority Garante has asked Facebook to provide it with clarifications related to its newly launched smart glasses, to determine the product's compliance with that nation’s privacy laws. Developed with Ray-Ban manufacturer EssilorLuxottica, the Ray-Ban Stories glasses allow users to hear music, take calls, or shoot photos and videos and share them across Facebook's services via a companion application. Garante made its request via the Irish Data Protection Commissioner (DPC), which oversees Facebook because the social-media giant's European headquarters are based in Ireland. The regulator said it wanted clarification on measures Facebook has deployed to shield people occasionally filmed, especially children, and on systems adopted to anonymize collected data, and features of the smartglasses' voice assistant.  ... ' 

Facebook Roy-Ban Smart Glasses

 Happened on this, more on a Facebook Smart Glasses.  Facebook Voice Assistant?  Hmmm ..  Too much in your face all the time digital world?  

Facebook and Ray-Ban Debut ‘Stories’ Smart Glasses With Facebook Voice Assistant

ERIC HAL SCHWARTZ on September 9, 2021 at 1:00 pm

Facebook and Ray-Ban have released a new line of smart glasses called Ray-Ban Stories controlled by the Facebook Assistant voice assistant. The two companies have been rumored to be collaborating on voice-controlled smart glasses, although the resulting focus on taking photos and videos is not what was anticipated.  ... '

Forward to Smart Glasses

 Next step forward to usable smart glasses, driven by tiny lasers.  General or very narrow applications?

Tiny Lasers Could Finally Bring Us Really Smart AR Glasses Post–Google Glass, a new alliance says it has the winning tech KATHY PRETZ    in Spectrum IEEE

When Google Glass debuted almost a decade ago, augmented reality (AR) wearables seemed poised to take off. Smart glasses let users surf the Internet; access maps, calendars, and other apps; and even call up recipes. Users could simultaneously interact with the physical and digital worlds.

But Google withdrew its smart glasses from the market in early 2015. Several reasons have been given for their demise, including poor marketing, an unattractive design, too many software glitches, and a short battery life.

"The technology Google Glass needed [to succeed]—such as illumination sources, compact projectors and optics, and a whole host of other things—was not mature enough at that time to meet all the requirements of design, performance, and use," says Bharath Rajagopalan, director of strategic marketing for STMicroelectronics, a multinational semiconductor manufacturer and a leader in microelectromechanical system (MEMS) technologies. "Sometimes when you're too early to the market, you don't have all the necessary elements."

Since the demise of Google Glass, companies have attempted to improve the design, performance, and functionality of smart glasses. The newer products include Blade glasses by Vuzix and Focals by North, which Google acquired last year.

Meanwhile, glasses with audio features only, including Bose Frames and Amazon Echo Frames, have sprung up. Facebook recently announced a collaboration with Luxottica on its version of audio-only smart glasses.

Some of the major players in the field say that for smart glasses to become more popular, they must be lightweight, low-power devices that are fashionable. A key enabling technology, they say, is laser-beam scanning (LBS) solutions, which ST has been a pioneer in developing.

LBS is gaining adoption for use in smart glasses because it uses a compact projector, produces a bright image with rich colors, consumes relatively little power, and can be integrated aesthetically into a pair of glasses, Rajagopalan says. Tiny MEMS mirrors create images by deflecting the laser beams emitted from compact diodes. They project the images onto waveguides or other combiner optics, he says. The combiner optics are typically embedded in smart-glasses lenses, onto which the images are projected, Rajagopalan says. Focals by North glasses use LBS technology.

Rajagopalan says the micromirrors and other tiny components—including lasers, projectors, and batteries—can all be housed unobtrusively in a module small enough to fit inside the frames. ....  '

Amazon Echo Tries Smart Glasses

A number of companies have tried the wearable assistant idea, announced originally in December, now Amazon will try the idea in earnest.  Will the idea work this time.  What key aspects are important?  In some tests we did with Google version, there was concern about how people using the glasses were perceived.  Were they continually taking pictures?    Is the idea for continuous use, which Amazon appears to imagine, or just for specialized assistant tasks?  We tried in plant maintenance tasks.   What would the key skills for glasses look like? 

Amazon Echo Frames arrive in blue light and sunglasses forms for summertime  in DigitalTrends  By Patrick Hearn  May 18, 2021 

The Amazon Echo Frames originally launched in December but came with a somewhat limited series of options. Today, Amazon announced two new models of the Echo Frames: A set of polarized sunglasses and a set of blue-light filtering glasses. The new options come after a lot of customer feedback asking for different options and styles.

The sunglasses themselves are available in two different versions, too. One is a classic set of sunglasses, while the other is a set of blue mirror sunglasses. Both versions are IPX4-rated, which means they are perfect for wearing while working out or walking outside. While they shouldn’t be worn while swimming, they can withstand a few splashes of sweat or rain.

On the other hand, the Echo Frames with a blue-light filter are perfect for people that spend a lot of time at their computers. Blue light can interfere with your circadian rhythm and interrupt your sleep cycle, so the blue light filter makes it possible to play video games or browse the web in the evenings without worrying about disturbing your rest.

The polarized blue mirror sunglasses will start shipping on May 18, while the Echo Frames with blue-light filtering lenses and the polarized classic sunglasses begin shipping on June 9. Both options are available in the classic black frames for $270.

The Echo Frames are more than just glasses. They provide access to Alexa with a touch of a button, as well as give you easy access to music, podcasts, and more without the use of headphones. You can control your music and playlists by saying “Alexa, pause,” or “Alexa, next.”  ...  "

Facebook Wants Watch like Controllers for AR Glasses

 Is Facebook aiming to take us long and deep with augmented reality controlled from our watches?    I still think there is something missing to make the idea universal.

Facebook reveals new watch-like controllers for its future AR glasses

The wearable devices will read nerve signals from the brain and deliver tactile feedback through haptic buzzers around the wrist.... 

By Mark Sullivan   in FastCompany

While other tech companies tend to keep their R&D under a blanket of secrecy, Facebook is opening its labs and showing the world how it’s developing its future augmented reality (AR) glasses. The company held the second in a series of its “Inside the Lab” media roundtables on Tuesday and introduced new technology that would enable users to control their AR glasses using their fingers.

AR glasses project light onto the eyes and seem to overlay the real world with digital imagery. As you’re looking through the glasses at a statue, for instance, the glasses might display a label with helpful information right next to it. Many people in tech circles believe AR glasses, in some form, will eventually replace the smartphone as our go-to personal computing device.

In this week’s roundtable, company executives, starting with chief technology officer Mike Schroepfer, showed off wrist wearables that detect the movement of the glasses wearer’s fingers. The wearable uses a technology called electromyography (EMG), which intercepts the electrical signals the brain uses to direct finger movements. It then translates the electrical pulses into digital commands that can control functions of the device functions. ... "

Pin Screens to the Walls

Is it the future of display to have information pinned on the wall.  Or floating in space?

Qualcomm’s new AR ‘Smart Viewer’ lets you pin virtual screens to your walls

Lenovo adopted the design for its ThinkReality glasses

By Adi Robertson@thedextriarchy  in TheVerge

Chip maker Qualcomm has introduced a new reference design for augmented reality glasses: an AR “smart viewer” you can tether to a phone or PC via USB-C. Called the XR1 Smart Viewer, the system is meant to be lightweight and look (sort of) like sunglasses, while also enabling features like hand tracking and spatial awareness. The first glasses based on its design are set for release in mid-2021. .. "

The State of Virtual Reality Hardware

Well done piece, suggesting we will soon be there for general usage wearable screens for our eyes to give the illusion of three dimensional immersive experiences. Saw examples in industry in the 80s,  is it ready?  Leading to 'smart glasses', or even embeddable devices.  

The State of Virtual Reality Hardware,  By Logan Kugler in   Communications of the ACM, February 2021, Vol. 64 No. 2, Pages 15-16   10.1145/3441290

For decades, virtual reality (VR) has seemed like a futuristic dream that is just around the corner, but never reaches its full potential. This time, however, might really be different. Recent advances in the power of VR hardware, notably the headsets and processors used to produce realistic VR experiences, suggest that VR is finally powerful enough and cheap enough to go mainstream.

VR broadly refers to immersing yourself in a three-dimensional (3D) digital world using sophisticated hardware and software. While a video game is experienced through a screen, VR often is experienced through a headset that shuts out the external world and transports you to a virtual one. It can also be experienced through room-sized systems that use special projectors and glasses to create VR experiences.

Historically, VR has relied on clunky headsets, expensive computers, and complicated peripheral hardware to produce immersive experiences. VR in various forms has been commercially available since the 1990s, but the technology has been widely criticized as too expensive, too complicated, or too imperfect to produce powerful, affordable virtual experiences that inspire consumers to open out their wallets.

That is beginning to change. Today, powerful commercial VR headsets are sold by Sony, Facebook, HTC, and other major technology players. Sophisticated augmented reality (AR) devices (like your smartphone and Google Glass) are available from the likes of Google, Apple, and Microsoft. The market for VR is growing accordingly, with research firm Marketsand-Markets forecasting industry growth to reach $20.9 billion in 2025, from $6.1 billion in 2020.

Why is VR (finally) having its day in the sun?

It all comes down to better hardware. VR heavyweights now are able to produce headsets that are cheaper and more powerful than models from just a few years ago. As a result, consumer demand for headsets is rising, driving more innovation and investment in VR hardware. Companies are even researching entirely new techniques and designs to make the next generation of VR hardware so light and powerful that it transforms one's daily life.

"Over time, we would like a device that is nearly the size of your reading glasses or sunglasses, but performs all of the functions of your smartphone, tablet, PC, and even TV, and enables new, 3D (three-dimensional) and spatial functions," says Siddharth Saxena, founder and CEO of Oblix VR, a VR software startup.

That day might not be far off.  .... " 

Amazon Smart Glasses are for Sale

Recall I have been following the idea of 'smart glasses' for a long time.   Since we experimented very early on with prototype glasses to display maintenance diagrams and data in real time on the plant floor.  That were never adopted,  they were slow to update and not easy to use.  But that was in the 80s.  I don't think these would work for that project either as they exist.

Amazon's Alexa-powered smart glasses are now more widely available

The $250 Echo Frames are Amazon's take on smart glasses.

By Nathan Ingraham in Engadget

Last year, Amazon announced Echo Frames, which are almost exactly what they sound like — a prescription-ready eyeglasses frame that looks pretty standard but has built-in microphones for you to summon Alexa. They were announced as a “Day One” product, made in limited quantities and only available via a special invitation. But now, a little over a year later, Echo Frames are available to anyone who wants them.

As the first Day 1 item graduating to a full-fledged product, Amazon has made a bunch of improvements to these new Echo Frames. The first model had open-ear speakers where you would hear replies, and the new model can automatically adjust volume based on how loud your surroundings are. Amazon also says battery life is up to 40 percent longer (during continuous playback at 60 percent volume). Another battery improvement comes from an “auto-off” features — if you put the frames upside-down for three seconds, they’ll automatically power down. ... ' 

Huawei's Wearable Smart Glasses

A look at another set of Smartglasses.  Good in depth piece.  As it says this is a difficult product to make well, especially as a general solution to effective and unobtrusively 'wearing' your computing.   Have followed the idea for some time, including specific early industrial applications.

Huawei’s Eyewear 2 smartglasses are like true wireless headphones for your face   By Andy Boxall in DigitalTrends

Huawei, in the face of challenges in the smartphone space, is exploring one of the more difficult products in the wearables space, smartglasses, to propel the brand forward. It’s new offering is the Huawei X Gentle Monster Eyewear 2 — and it’s about as risky as tech products get.

Wearables, in particular smartglasses, need to enhance your life without being intrusive. It’s a simple formula, but one that’s very hard to get right. Not being intrusive means having a design that’s as “normal” as possible, and that’s hard to do when something needs to attach to our face — and inevitably still require everything from Wi-Fi and Bluetooth radios to batteries inside.

I’ve worn the Eyewear 2 smartglasses for a short time, so did I see the future, or the ghost of Google Glass?  ... '

Better VR Hearing

Adding additional sensors in AR cold lead to specific new application areas.  Spatial Audio could add to maintenance capabilities when linked to machine learning.  Also that Facebook continues to be interested in 'smart glasses' direction.  Perhaps not superhearing though. 

Facebook wants its AR glasses to give wearers superhearing

It could improve VR headset audio, too

By Adi Robertson@thedextriarchy  in TheVerge

 A new Facebook experiment shows how sound could play a major role in augmented reality. Chief scientist Michael Abrash and his team at FRL Research (formerly Facebook Reality Labs) released details today https://about.fb.com/news/2020/09/facebook-reality-labs-research-future-of-audio/  about what the team calls “perceptual superpowers” — AR systems that figure out what you’re trying to hear, then amplify it and dampen background noise. Combined with spatial audio features, the device creates the aural equivalent of a hologram overlay in a pair of glasses.  ... " 

Korean Mixed Reality Glasses

Another attempt at more generalized smart glasses, here aimed at mixed reality in particular.  Casual everyday wearable, or context specific wearable for the job. linked closely to mobile.

Nreal Light mixed reality glasses launch in Korea with the Galaxy Note 20
LG Uplus is bundling the Nreal Light with a Samsung Galaxy Note 20 or an LG Velvet.

By Richard Lai, @richardlai in Engadget

After successfully fighting off Magic Leap’s claim that it stole trade secrets, Nreal is finally able to launch its Light mixed reality glasses into the consumer market. Starting today, folks in Korea can pre-order the Light — locally rebranded as “U+ Real Glass” — as part of a mobile phone plan on the LG Uplus network, so long as you pick the Samsung Galaxy Note 20 or the LG Velvet as your handset. That way you can buy the Light at a subsidized price of 349,500 won (about $295). You can also purchase the glasses separately for 699,000 won (about $590) at an LG Uplus store from August 21st, if you’d rather use them with other phones. ... "

Google Pushing Advanced Wearables

Good to see Google pushing this research. Still consider the use of this kind of focused wearable to have narrow uses, until people decide it to be part of their normal clothing.

Google is quietly experimenting with holographic glasses and smart tattoos
The search giant has been working on or funding a new generation of wearable technology.
Richard Nieva in CNET

A simple pair of sunglasses that projects holographic icons. A smartwatch that has a digital screen but analog hands. A temporary tattoo that, when applied to your skin, transforms your body into a living touchpad. A virtual reality controller that lets you pick up objects in digital worlds and feel their weight as you swing them around. Those are some of the projects Google has quietly been developing or funding, according to white papers and demo videos, in an effort to create the next generation of wearable technology devices.

The eyewear and smartwatch projects come from the search giant's Interaction Lab, an initiative aimed at intertwining digital and physical experiences. It's part of Google Research, an arm of the search giant with roots in academia that focuses on technical breakthroughs. The Interaction Lab was created within Google's hardware division in 2015, before it was spun out to join the company's research arm about two years ago, according to the resume of Alex Olwal, the lab's leader. Olwal, a senior Google researcher, previously worked at X, the company's self-described moonshot factory, and ATAP, Google's experimental hardware branch. 

The goal of the Interaction Lab is to expand Google's "capabilities for rapid hardware prototyping of wearable concepts and interface technology," Olwal writes. Its initiatives appear to be more science experiment than product roadmap, with the likely goal of proving ideas rather than competing with the Apple Watch or Snapchat Spectacles. But taken together, they provide a glimpse at Google's ambitions for wearable tech.

The other projects were collaborations with researchers from universities around the world. At least two of them -- the VR controller and smart tattoos -- were partly funded through Google Faculty Research Awards, which support academic work related to computer science and engineering. The efforts highlight Google's close ties with the academic community, a bridge to the company's beginnings as a Stanford University grad school project by co-founders Larry Page and Sergey Brin that grew into a global behemoth with deep hooks into our lives. .... "

Alphabet Buying North Glasses

Somewhat Unexpected perhaps,  but some indication of seriousness on the ultimate wearable. Had heard from others that North Glasses were good.

Report: Alphabet looking to buy smart glasses startup North for $180M   By  Maria Deutscher

Google LLC parent Alphabet Inc. is reportedly in advanced talks to acquire North Inc., a Canadian smart glasses startup, for about $180 million.

The discussions were first reported by Canada’s The Globe and Mail on Thursday evening. The publication’s sources indicated that Ontario-based North is in the “final stages” of hammering out an agreement with Alphabet.

Founded in 2012, North is backed by about $160 million from investors that include Intel Corp.’s venture capital arm and Salesforce.com Inc. founder Marc Benioff. The startup offers a smart glasses product called Focals that’s touted as less bulky than many other products in the category, closely resembling a regular pair of glasses. Users can view mobile notifications without looking at their phone and perform certain other tasks such as ordering an Uber ride.  ..."