/* ---- Google Analytics Code Below */
Showing posts with label AR. Show all posts
Showing posts with label AR. Show all posts

Sunday, June 25, 2023

Speech AI Spotlight

 Would expand potential capabilities for AR ... makes me broaden they potential usefulness.

Speech AI Spotlight: Visualizing Spoken Language and Sounds on AR Glasses

Jun 23, 2023

By Sirisha Rella

Audio can include a wide range of sounds, from human speech to non-speech sounds like barking dogs and sirens. When designing accessible applications for people with hearing difficulties, the application should be able to recognize sounds and understand speech.

Such technology would help deaf or hard-of-hearing individuals with visualizing speech, like human conversations and non-speech sounds. Combining speech and sound AI together, you can overlay the visualizations onto AR glasses, making it possible for users to see and interpret sounds that they wouldn’t be able to hear otherwise. 

According to the World Health Organization, about 1.5B people (nearly 20% of the global population) live with hearing loss. This number could rise to 2.5B by 2050.

Cochl, an NVIDIA partner based in San Jose, is a deep-tech startup that uses sound AI technology to understand any type of audio. They are also a member of the NVIDIA Inception Program, which helps startups build their solutions faster by providing access to cutting-edge technology and NVIDIA experts.

The platform can recognize 37 environmental sounds, and the company went one step further by adding cutting-edge speech-to-text technology. This gives a truly complete understanding of the world of sound.

AR glasses to visualize any sound

AR glasses have the potential to greatly improve the lives of people with hearing loss as an accessible tool to visualize sounds. This technology can help enhance their communication abilities and make it easier for them to navigate and participate in the world around them.  ... ' 

Tuesday, June 06, 2023

Vision Pro: Apple's New Augmented Reality Headset Unveiled

Pricey indeed, likely first specialty applications.

Vision Pro: Apple's new augmented reality headset unveiled    By Tom Gerken & James Clayton at Apple Park in California    in BBC News

Apple has unveiled a much-anticipated augmented reality headset, Apple Vision Pro, in its first major hardware launch for almost a decade.

Apple CEO Tim Cook said the new headset "seamlessly blends the real world and the virtual world".

The tech firm also announced its latest iPhone operating system, as well as updates to MacBook Air.

The headset has a two-hour battery life, costs $3,499 (£2,849) and will be released early next year in the US.

The cost is considerably more than virtual reality headsets currently on the market. Last week Meta announced its Quest - which costs $449.

Apple said little about generative artificial intelligence - the buzzy technology that is the talk of Silicon Valley.

The company's share price fell slightly during the announcement, made at a developer's conference at Apple Park, the company's headquarters, in Cupertino, California.

The BBC is among the media outlets at the event, but has yet to try out the new device.

'Ski goggles'

Apple Vision Pro looks different to similar headsets on the market - and is more reminiscent of a pair of ski goggles than a virtual reality headset.

Apple used the phrase "augmented reality" to describe what the new device does.

Augmented reality, also known as mixed reality, superimposes virtual objects in the world around us - enabling us to mix reality with virtual reality by looking through a screen.

Users can access apps, watch movies, and write documents in a virtual world. But so far, there is little evidence of a big market for this kind of wearable tech.

Hartley Charlton, senior editor of MacRumors, was unsure how much the headset would appeal to the general public.

"It won't appeal to mainstream consumers at first on account of its extremely high price point and immediate shortcomings as a first-generation device," he said, "such as its separate wired battery pack."

But he said Apple has a track record of "overcoming scepticism" about new devices, and has historically encouraged people to "part with their cash to add a new gadget to their repertoire".


Friday, April 28, 2023

AR Art Takes Over British City

A novel idea. 

AR Art Takes Over British City

Wired, Elissaveta M. Brandon, April 24, 2023

The U.K. city of Sheffield has transformed its rooftops into an augmented reality (AR)-based art display. Launched in February, the "Look Up!" project is an "art trail" comprised of four buildings in the city's center, each coupled to a quick response (QR) code on the sidewalk below. Viewers can use a free smartphone application to scan the codes and follow animated arrows directing their gaze upward, to watch a stick figure made of multicolored balloons, a giant cat, and other whimsical animated characters onscreen. U.K. company Megaverse created the app and platform in partnership with U.S.-based AR developer Niantic, while local firms Universal Everything and Human Studio supplied the artworks. More than 1,500 people had downloaded the app and nearly 2,000 QR codes had been scanned in the week following the trail's launch.  ... 


Thursday, March 02, 2023

Augmented Reality Headset Enables Users to See Hidden Objects

Who decides what we get to See? 

Augmented Reality Headset Enables Users to See Hidden Objects

By MIT News, March 1, 2023 

The X-AR augmented reality headset combines computer vision and wireless perception to automatically locate a specific item that is hidden from view, then guide the user to it.

Credit: MIT News

Massachusetts Institute of Technology researchers have combined computer vision and wireless perception into X-AR, an augmented reality (AR) headset that can visualize hidden objects.

X-AR employs radio frequency (RF) signals to find concealed items with RF identification tags; those items are represented by a transparent sphere to guide headset wearers to their locations.

The system confirms that users have retrieved the correct object once it is in their hand.

Tests in a warehouse-like setting showed X-AR could localize hidden items to within 9.8 centimeters (3.8 inches) on average, while verifying that users retrieved the correct item 96% of the time.

From MIT News


Thursday, February 02, 2023

You Might be Able to Write Your Own Apple AR/VR Headset Apps?

Like the idea, But how easy will it be?   Lots of dated rumor out there, warning. 

You might be able to write your own Apple AR/VR headset apps

Includes link to a podcast  in TechRadar,  By David Nield 

The long-rumored Apple AR/VR headset might possibly, at last, be launching later this year – and we've got a new set of leaks around app development for it, hinting that everyone will be able to create augmented reality experiences on this device.

According to sources speaking to The Information(opens in new tab) (via MacRumors(opens in new tab)), Apple is working on software development tools to make the app creation process as straightforward as possible. The more apps get built, the more appealing the headset will be.

These tools mean that even consumers could build their own apps through Siri, the report says: so you could just describe the experience that you wanted to see in augmented reality, and Siri would interpret your commands and build it for you.

One of the examples mentioned in the article is being able to build an app showing virtual animals walking around a room, just by describing it to Siri. No coding or computer graphic design skills would be required to put it together.

It's worth noting that the information sourced in the story is from 2021, so it might be a little outdated at this stage – and it's obviously unconfirmed and unofficial to begin with. However, it's logical that Apple would want to make app development as easy as it can.

The report also mentions that Apple is working on AR content for the headset, with health and wellness a priority. One potential use of the device is as a meditation aid, enabling users to sit in digitally generated, calming scenes of their choosing.  ....  ' 

Sunday, December 18, 2022

Mobile Augmented Reality Talent Sources

Agree, this is not common, perhaps in the game space?   Appreciate the lack of connective knowledge.   We spent some time at this in two very different domains.  (Defense and Packaged Goods) 

The talent Needed to adopt mobile AR in industry    in ChiefLeaningOfficer

Chief Learning Officer Network by Elizabeth Loutfi / December 12, 2022

Over the past two decades, mobile and augmented reality technologies have been introduced into the design, assembly, maintenance, quality and logistics sectors to assist job performance.

Mobile AR technology is used to assist training and support job performance during a task. For example, mobile AR was used in training technicians’ maintenance and assembly skills, such as a worker using AR on a tablet to perform tasks with a piece of a machine.

Meanwhile, mobile AR can also support mobile workers in completing job tasks by accessing and processing information during their work and with the minimum help from others. For instance, according to a study on AR in warehouse operations, warehouse workers reduced errors in operations such as receiving, storing, picking and shipping with AR on wearable devices.

Additionally, many pilot studies and field tests revealed the positive results that mobile AR brought to companies and organizations, such as improved job performance, reduced time on tasks and errors, and increased confidence. However, the talent required for learning and development professionals to successfully adopt mobile AR in the workplace remains unknown. 

This article presents four emerging findings from a recent study about the talent learning and development professionals needed to adopt mobile AR in the industry. The participants came from North American companies and organizations, representing industries such as supply chain, biomedical, utility, electronics engineering, manufacturing and AR consulting services.

Instructional design knowledge and skills

Instructional design knowledge and skills are usually required when developing training programs in different modalities, such as face-to-face, online or blended. When adopting mobile AR technology to drive job performance, both industry and consulting service participants recognized the importance of “good instructional designs and learning development capability.”  

Furthermore, instructional design knowledge and skills are interpreted in two types: The first type refers to the ability of developing the content to design effective learning or job performance with AR. A CEO from an AR consulting service suggested that it was necessary “to identify a clear performance objective and to break down” based on the criterion to avoid creating a “lot of content, a lot of media and stuff, and then it’s not well received.” 

The second type refers to the ability of understanding needs and business problems. A learning director from the supply chain industry commented that “most of the consultants are coming to us with technology without understanding the problem.” He viewed that “technology should be the enabler. It should not be the driver of the solution.”

Mobile AR technical skills

Mobile AR technical skills refers to the ability to use a tool or a system to develop AR content and deploy it to end users. It is similar to a content development tool or learning management system in the e-learning industry.

The views about mobile AR technical skills were very mixed. A project manager from an electronics engineering and manufacturing industry thought the current AR tool was very powerful but complex. She suggested spending time learning the AR tool. However, several participants from the industry and AR consulting services viewed the current AR tools as easy to use, just like developing PowerPoint.

Overall, participants agreed that the better the AR authoring tool, the less L&D professionals should worry about AR technical skills. More attention should be focused on design, such as whether to develop content for learning or supporting job performance.   ... 

Tuesday, December 13, 2022

Big Players Ready for Meta Headset Offerings

Big Players and more Meta Reality Hype and Prep

Apple Is Now Calling Its AR/VR Headset Operating System ‘xrOS’

By Josh Norem on December 5, 2022  in ExtremeTech

Apple seems to be getting close to the launch of its much-hyped AR/VR headset, as it’s reportedly settled on a name for the operating system. Noted Apple soothsayer Mark Gurman is reporting it’s going to be xrOS. This is shorthand for “extended reality,” as opposed to strictly augmented reality (AR) or virtual reality (VR). Previously it was believed to be called realityOS, but Apple has allegedly ditched that for a shorter name, the report said. (The Bloomberg article is paywalled and was summarized by MacRumors.)

The device will be Apple’s first new product since the Apple Watch, which debuted in 2015. The new naming confirms Apple’s plans to follow in the footsteps of the Meta Quest Pro by offering mixed, or extended, reality. That approach will allow the user to experience both VR, a fully digital world, and AR, virtual objects overlaid with reality. This is seen as a more flexible offering than just sticking with one technology. Current headsets like the Meta Quest 2 and Valve Index are strictly VR headsets. ... ' 

Sunday, September 25, 2022

Cognixion One, a Brain Interface with AR

Brought to my attention.    Updates to Cognixion.   More at the link. Let me know of your experience and I will report it here ....

CXN ONE

The World's First Brain Computer Interface with Augmented Reality

Wearable Speech™ Generating Device   ©2020 Cognixion, Patents Pending

Say Hello to Cognixion ONE​  ...  (link above) 

For too long, the assistive technology industry has relied on repurposed consumer electronics, often years behind the cutting edge of what is possible.

At Cognixion, we believe that every individual deserves a solution as unique as they are – and that it’s possible to build one tool for communication, access, and everything else life brings their way.

Enter Cognixion ONE. No more dangling wires. No more PC monitor blocking your view of the person you’re speaking with. No more leaving your voice and your control at home, or in the classroom; Cognixion ONE is a wearable window to the world, offering both speech and an integrated AI assistant for home automation control and other enrichment. 

Whether you use Cognixion ONE’s AR environment via head pointing, Brain-Computer Interface, or switch, you’ll see how it’s built from the ground up to reduce the lag between intention and outcome.

Think it. Say it. Do it.

Cognixion ONE - The world's first wearable brain computer Interface with AR | Product Hunt

Extending Abilities to the Most Vulnerable

Designed by neurologists and biosignal engineers in concert with Speech-Language Pathologists and a large group of users and professionals, Cognixion ONE comes out of the gate with integrated home automation AI and multiple context-aware predictive keyboards. We’re prepared to meet the needs of anyone with complex communication disorders including CP, ALS, and hundreds of other conditions exhaustively researched and tested by our team.

Almost anyone who has been reliant on switch or eye control access is an immediate candidate for Cognixion ONE, as well as many other conditions for which there was previously no good access method such as Locked-In Syndrome or hyperkinetic conditions causing too much movement for reliable eye tracking. Our durable design and optional accessory straps mean you can use Cognixion ONE with confidence all throughout your daily life.

Your Window to the World

Cognixion ONE is an entirely wireless, mobile solution that doesn’t just give you freedom to move – it gives you freedom to move the world. 

Direct integration with a popular AI assistant (tba) provides access to home automation, music, games, and more; you don’t need to own a hub, because it is a hub. 

Our speech generating software also displays text on the mirrored lens, so you can be understood even in loud environments – or disable sound and use text only for private conversations. 

A 4G cellular connection ensures you always have a direct line to your AI assistant and our other cloud services, all of which are completely secure and compliant with all international privacy laws.  ... '

Tuesday, September 20, 2022

Walmart Tries out AR with Machine Learning for Clothing Trial

Experimented with this and related cosmetics approaches.  

Walmart Rolls Out AR to Try on Clothing Virtually

By Fast Company, September 19, 2022

Walmart has launched a new augmented reality feature that allows shoppers to virtually try on clothes.

The Be Your Own Model tool uses machine learning technology originally developed for topographic mapping so customers can view how apparel will fit on their bodies, along with shadows, colors, and simulated fabric draping.

Be Your Own Model runs on Walmart's iOS application, with shoppers able to upload photos to model the apparel on images of their bodies.

Walmart's Cheryl Ainoa said the process is energy efficient so customers will not run down their phones' batteries.

From Fast Company

View Full Article   

Monday, August 08, 2022

VegSense Gathers Forestry Data in AR

Another idea that permits a level of continuous sensing of information by professionals for a rich scan. 

VegSense Makes Sense for Forest Studies

Rice University News, Mike Williams, August 1, 2022

Rice University bioscientists combined a mixed-reality Microsoft HoloLens headset with the open-source VegSense software to measure vegetation for forest studies. Rice's Daniel Gorczynski said the hardware-software combination is less expensive than light detection and ranging (LiDAR) systems most frequently used in three-dimensional field studies. The headset displays a grid pattern that tracks the surfaces of vegetation as the user scans the environment. Gorczynski said the object "is to get the mesh to cover as much of the vegetation as possible because that's what gets you the best scan." In testing, VegSense detected 48 of 50 relatively mature trees within a circle about 30 feet in diameter in Houston's Memorial Park. ... 

Deaf See Conversations via Subtitles

The general idea fairly obvious,   How well is important. 

AR Glasses Allow Deaf People to 'See' Conversations by Turning Audio into Subtitles

EuroNews, Aisling Ní Chúláin, July 29, 2022

U.K. startup XRAI Glass has unveiled new augmented reality (AR) glasses that can translate audio into subtitles for deaf and hard-of-hearing users. The XRAI Glasses, developed with Chinese AR glasses manufacturer Nreal, are connected to a mobile phone that manages processing and graphics generation. "What our software effectively does is it takes an audio feed from the microphone on these glasses [and] sends it down to the phone," said XRAI Glass' Dan Scarfe. "On the phone we are effectively turning that audio into closed captions and then, using that Nreal software, we can project those subtitles into the real world." ...

Tuesday, August 02, 2022

Euro Glasses

 Had not seen this as yet, note these are developed with Chinese manufacturer Nreal. 

AR Glasses Allow Deaf People to 'See' Conversations by Turning Audio into Subtitles

EuroNews

Aisling Ní Chúláin, July 29, 2022

U.K. startup XRAI Glass has unveiled new augmented reality (AR) glasses that can translate audio into subtitles for deaf and hard-of-hearing users. The XRAI Glasses, developed with Chinese AR glasses manufacturer Nreal, are connected to a mobile phone that manages processing and graphics generation. "What our software effectively does is it takes an audio feed from the microphone on these glasses [and] sends it down to the phone," said XRAI Glass' Dan Scarfe. "On the phone we are effectively turning that audio into closed captions and then, using that Nreal software, we can project those subtitles into the real world."

Full Article

Thursday, July 28, 2022

Google AR Glasses Update

 Will this open the door to new augmented reality AR?

Google Starts Real-World Testing for Augmented Reality Glasses That May Employ North Focals Tech

ERIC HAL SCHWARTZ on July 21, 2022 at 7:00 am in Voicebot.ai

Google will start real-world testing of augmented reality glasses for translation and navigation next month. Several dozen testers will walk around wearing prototypes AR glasses with inset lens displays, microphones, and cameras for limited activites like translating text and providing audio directions. The description suggests the device extends and improves on the technology Google acquired in 2020 from North, the makers of Focals by North smart glasses.

GOOGLE AR

Google has spent years developing AR devices, code-named Iris, according to some rumors. Recent reports suggest a 2024 release for the device, which may hide its tech enough to look like regular prescription glasses, a goal North was working toward before getting acquired. The upcoming tests are circumscribed to things like translating menus and displaying directions. For privacy, the prototypes won’t even allow for photos or video recordings despite the available hardware.

The version Google releases will likely be more ambitious in employing video and photographic services. The microphone will presumably connect wearers to Google Assistant, though the operating system is still unknown. Google’s vice president of Labs Clay Bavor is still running the AR and VR division, following his Google Cardboard and Daydream VR platform development and is said to be working with Google Assistant creator Scott Huffman on the project too. That Google is moving to tests beyond controlled environments is certainly an encouraging sign for those hoping to see a device that will permanently wipe out bad memories of Google Glass.   .... ' 

Saturday, June 25, 2022

A Future for AR

 Like the proposed connection to optimization ...   visualizing the value always useful. 

Intensity Control of Projectors in Parallel—Doorway to an Augmented Reality Future

Tokyo Tech News, March 16, 2022

Scientists at Japan's Tokyo Institute of Technology (Tokyo Tech) have developed a method for interacting with dynamic objects in augmented reality (AR), while reducing latency. Dynamic projection mapping combines cameras and projectors that respectively detect and project onto target surfaces, and the researchers' technique can calculate the intensity of each pixel on a target in parallel. This reduces the need for a single large optimization calculation, significantly boosting mapping speed and shortening latency. "The presented high-speed multi-projection is expected to be a major part of important base technologies that will advance spatial AR to derive more practical uses in our daily life," explained Tokyo Tech's Yoshihiro Watanabe. .... ' 

Sunday, May 15, 2022

How AR and VR are Changing Customer Experience

How AR and VR are transforming customer experiences  in Venturebeat

AR and VR technology was largely expedited by the past pandemic with at least 93.3 million and 58.9 million users respectively, according to a study conducted by eMarketer. This comes as no surprise since these immersive technologies have mitigated the massive disruptions in people’s lives. 

Their accelerated adoption at both individual and business levels are represented in daily activities. For example, as remote work became an essential part of many business models, AR/VR allowed a seamless transition from onsite training to a clear visualization of step-by-step instructions for many workers across multiple industries and locations.   ... '

Monday, May 02, 2022

NReal Glasses

 Still awaiting these,

NReal AR glasses set to launch this spring in UK

Details on a release date of the product’s release or pricing remain unclear

Chinese augmented reality (AR) startup Nreal will be bringing its smart glasses to the U.K. this spring. 

British mobile operator EE, which is owned by telecoms group BT, confirmed the news Tuesday, saying it has partnered with the Beijing-based company to "exclusively bring the Nreal Air AR glasses to the UK’s best network for 5G." ... ' 

Wednesday, April 13, 2022

Innovative, Practical Applications of the Metaverse: Podcast and Transcript

 Good overview,   Note it points to past pieces. 

Innovative and Practical Applications of the Metaverse

March 29, 2022 | Podcast  with Richard Ward, Mina Alaghband  of McKinsey

The immersive experience of the metaverse allows for massive shifts in training, education, and work. In this episode of the At the Edge podcast, McKinsey expert Richard Ward speaks with McKinsey’s Mina Alaghband to share the myriad ways in which metaverse technology is already being implemented and how executives can get started to discover where the business opportunities exist, including no-regret moves they can make today.

An edited transcript of the discussion follows. For more conversations on cutting-edge technology, follow the series on your preferred podcast platform.

Podcast transcript:

When it comes to your organization, what are practical ways to use the metaverse?

Why not practice first in the metaverse, where it’s low cost, you can do things in an infinite way, and you can make the impossible happen? Those types of things really offer people more efficiency and productivity.

Mina Alaghband: That’s Richard Ward, a McKinsey senior expert who joins me today. This is the last of a three-part series on the metaverse. I’m Mina Alaghband. Welcome to At the Edge, a production of McKinsey’s Technology Council.

Richard, thank you so much for joining us today. We’ve heard a lot about this ten-year aspiration for the metaverse to be a place where we all reside and have our identities, and there is a secondary economy that some say will be even bigger than the real-world economy. But business leaders will want to know what this looks like today, and what it will look like over the next few years. Could you start by describing some of the near-term use cases you’re seeing in the market?

Richard Ward: The things that you can do today are really quite interesting. On the VR [virtual reality] end of the spectrum, a lot of heavy industries and even the United States military have discovered that when you need to teach people a new vocational skill, like how to repair a piece of equipment such as a truck or a helicopter, one of the really long parts of that educational process is people learning where all the parts are on the equipment. So a number of organizations have decided to go VR and metaverse first, rather than real-world first.

It used to take 18 weeks to train a new recruit to work on one of these helicopters, but what they do now is hand a new recruit a pair of VR goggles, which in effect gives them their own private helicopter to learn on. They spend several weeks doing very intensive quizzes and simulations—where is the air filter and what order do you put these wires in? And one of the fascinating things that has come out of this is that they’ve been able to shorten their training course from 18 weeks down to ten weeks. This provides a real-world example of how if you need to do something that is very manual or requires you to move around a lot, metaverse tools can be very helpful. That’s one of the key elements of the metaverse: you can move around, you can walk places, you can see things, you can do things.

This technology is very powerful in terms of helping people learn things that are manual or tactile in the hand and that involve muscle memory. Much of the world does this type of work, and to be able to make the training for it more efficient and higher quality is a very exciting story.

Mina Alaghband: Can you give us an example in the financial services industry? What might an application of the metaverse look like today or in the near future?

Richard Ward: Whether you agree with the idea of NFTs—nonfungible tokens—the fact is that they are an asset class, and the financial services arena is starting to bring all of their usual skills and capabilities into this new asset class. There have now been loans that have been collateralized based on the agreed-to value of these NFTs. Similarly, there was a report recently about a firm issuing a mortgage for a virtual real estate deal in one of these digital, proto-metaverse environments.1 That’s essentially a collateralized loan, but still it’s very exciting that these kinds of concepts flow through this new digital setting.

The thing that adds a lot of interest to this is that the use of NFTs involves a lot of foreign exchange activity. So you have money going between, let’s say, US dollars into Ethereum, and then from Ethereum into another [crypto] coin or into other similar kinds of assets that are somewhat semifungible—those are activities that are very familiar to the financial services sector. It’s basically foreign exchange trading and management of risk. We’re still in the very early days, and I would defy anyone to truly be able to tell us about everything that financial services will be able to do five years from now.

Mina Alaghband: If I’m in the industrial sector, what kind of applications might I be seeing today? And are these examples of augmented reality [AR] or VR?

Richard Ward: In field operations, people are starting to use AR for remote assistance provided through a smartphone or a tablet or even a set of fancy glasses with special lenses in them.

The really interesting part about this is when you start using the data generated from this process. For example, you could ask, “What are the things that we’re getting the most remote assistance calls on?” and then flow that information into updating the training program. You could then use that data to update an assessment program to see if people really did learn the things they need to learn, and then potentially see the number of calls for a category of problems go down.  .... "

Friday, April 01, 2022

Training the Metaverse with Sim Eye Movements

Simulated Human Eye Movement Aims to Train Metaverse Platforms

Duke University Pratt School of Engineering, Ken Kingery,  March 7, 2022

Virtual eyes developed by computer engineers at Duke University could be used to train virtual reality and augmented reality programs for the metaverse. The program, called EyeSyn, accurately simulates how humans look at the world while also protecting user data. The researchers relied on cognitive science literature on how humans see the world and process visual information to develop the program. Tests showed that EyeSyn closely matched the distinct patterns of actual gaze signals and simulated the different ways people's eyes react. EyeSyn eliminates the need for companies building platforms and software in the metaverse to collect data on how peoples' eyes move during various activities. "If you give EyeSyn a lot of different inputs and run it enough times, you'll create a data set of synthetic eye movements that is large enough to train a (machine learning) classifier for a new program," said Duke's Maria Gorlatova. ... 

Thursday, December 30, 2021

Augmented versus Virtual Meta-Verses

 A space we also explored, in both senses.  Harder yes, but not sure the user cares if the value is provided.   And does not matter otherwise.  

Why AR, not VR, will be the heart of the metaverse

Join gaming leaders, alongside GamesBeat and Facebook Gaming, for their 2nd Annual GamesBeat & Facebook Gaming Summit | GamesBeat: Into the Metaverse 2 this upcoming January 25-27, 2022. Learn more about the event. 

This article was contributed by Louis Rosenberg, CEO and chief scientist at Unanimous AI

My first experience in a virtual world was in 1991 as a PhD student working in a virtual reality lab at NASA. I was using a variety of early VR systems to model interocular distance  (i.e. the distance between your eyes) and optimize depth perception in software. Despite being a true believer in the potential of virtual reality, I found the experience somewhat miserable. Not because of the low fidelity, as I knew that would steadily improve, but because it felt confining and claustrophobic to have a scuba mask strapped to my face for any extended period.

Even when I used early 3D glasses (i.e. shuttering glasses for viewing 3D on flat monitors), the sense of confinement didn’t go away. I still had to keep my gaze forward, as if wearing blinders to the real world. There was nothing I wanted more than to take the blinders off and allow the power of virtual reality to be splattered across my real physical surroundings.

This sent me down a path to develop the Virtual Fixtures system for the U.S. Air Force, a platform that enabled users to manually interact with virtual objects that were accurately integrated into their perception of a real environment. This was before phrases like “augmented reality” or “mixed reality” had been coined. But even in those early days, watching users enthusiastically experience the prototype system, I was convinced the future of computing would be a seamless merger of real and virtual content displayed all around us.

The 2nd Annual GamesBeat and Facebook Gaming Summit and GamesBeat: Into the Metaverse 2

January 25 – 27, 2022

Augmented Reality Research 1992 (USAF - L Rosenberg)

Cut to 30 years later, and the phrase “metaverse” has suddenly become the rage. At the same time, the hardware for virtual reality is significantly cheaper, smaller, lighter, and has much higher fidelity. And yet, the same problems I experienced three decades ago still exist. Like it or not, wearing a scuba mask is not pleasant for most people, making you feel cut off from your surroundings in a way that’s just not natural.

This is why the metaverse, when broadly adopted, will be an augmented reality environment accessed using see-through lenses. This will hold true even though full virtual reality hardware will offer significantly higher fidelity. The fact is, visual fidelity is not the factor that will govern broad adoption. Instead, adoption will be driven by which technology offers the most natural experience to our perceptual system. And the most natural way to present digital content to the human perceptual system is by integrating it directly into our physical surroundings.

Of course, a minimum level of fidelity is required, but what’s far more important is perceptual consistency. By this, I mean that all sensory signals (i.e. sight, sound, touch, and motion) feed a single mental model of the world within your brain. With augmented reality, this can be achieved with relatively low visual fidelity, as long as virtual elements are spatially and temporally registered to your surroundings in a convincing way. And because our sense of distance (i.e. depth perception) is relatively coarse, it’s not hard for this to be convincing.

But for virtual reality, providing a unified sensory model of the world is much harder. This might sound surprising because it’s far easier for VR hardware to provide high-fidelity visuals without lag or distortion. But unless you’re using elaborate and impractical hardware, your body will be sitting or standing still while most virtual experiences involve motion. This inconsistency forces your brain to build and maintain two separate models of your world — one for your real surroundings and one for the virtual world that is presented in your headset.

When I tell people this, they often push back, forgetting that regardless of what’s happening in their headset, their brain still maintains a model of their body sitting on their chair, facing a particular direction in a particular room, with their feet touching the floor (etc.). Because of this perceptual inconsistency, your brain is forced to maintain two mental models. There are ways to reduce the effect, but it’s only when you merge real and virtual worlds into a single consistent experience (i.e. foster a unified mental model) that this truly gets solved. .... '

Friday, December 10, 2021

Augmented Reality with Heads-up Displays

 Some of our earliest AR work was with heads up displays (HUDs) which replicated real maintenence environments.  Here they are dealing with automotive examples, where fast reaction and realist attention would be key.  Good detail here on important concepts.

The Road Ahead for Augmented Reality  By Keith Kirkpatrick

Communications of the ACM, December 2021, Vol. 64 No. 12, Pages 20-22  10.1145/3490317

Automotive head-up displays (HUDs), systems that transparently project critical vehicle information into the driver's field of vision, were developed originally for military aviation use, with the origin of the name stemming from a pilot being able to view information with his or her head positioned "up" and looking forward, rather than positioned "down" to look at the cockpit gauges and instruments. The HUD projects and superimposes data in the pilot's natural field of view (FOV), providing the added benefit of eliminating the pilot's need to refocus when switching between the outside view and the instruments, which can impact reaction time, efficiency, and safety, particularly in combat situations.

In cars, the main concern is distracted driving, or the act of taking the driver's attention away from the road. According to the National Highway Transportation Safety Administration, distracted driving claimed 3,142 lives in 2019, the most recent year for which statistics have been published. Looking away from the road for even five seconds at a speed of 55 mph is the equivalent of driving the length of a football field with one's eyes closed.

As such, the desire to ensure that drivers keep their eyes focused on the road, instead of looking down at the gauges on the dashboard, was the impetus for the development of HUDs suitable for use in production automobiles. The first automotive HUD that was included as original equipment was found on the 1988 Oldsmobile Cutlass Supreme and Pontiac Grand Prix; both were monochromatic, and displayed only a digital readout of the speedometer.

Thanks to the increasing inclusion of a variety of automotive sensors and cameras, advanced driver assistance system (ADAS) features and functions (such as automatic braking, forward collision avoidance, lane-keeping assist, and blind-spot monitoring, among others), and more powerful on-vehicle processors, automakers have been installing HUD units in commercial vehicles that provide more essential driving data, such as speed, engine RPMs, compass heading, directional signal indicators, fuel economy, and other basic information, allowing the driver to concentrate on the road instead of looking down to check the dash or an auxiliary screen.

The technology enabling most types of HUD is based on the use of a processor to generate a digital image of data coming from sensors. These images then are digitally projected from a unit located in the dash of the car onto a mirror or mirrors, which then reflect that image onto either a separate screen located behind the steering wheel, or onto the vehicle's windshield, directly in the driver's forward view. Common projection and display technologies used include liquid crystal display (LCD), liquid crystal on silicon (LCoS), digital micromirror devices (DMDs), and organic light-emitting diodes (OLEDs), which have replaced the cathode ray tube (CRT) systems used in the earliest HUDs, as they suffered from brightness degradation over time.

The HUDs that project the information onto a separate transparent screen are called combiner HUDs; these were popular because the physical space required to install the system was modest, and because the system was fully integrated, OEMs did not need to design a system that accounted for each vehicle's unique windshield angle or position. However, this type of HUD was limited by several factors; namely, the optical viewing path of a combiner HUD is shorter than looking through a windshield, and the driver's eyes must refocus slightly to the shorter visual distance when switching between looking out the windshield and checking the display. Furthermore, there is a practical limit to the size and field of vision (FOV) offered by combiner units; adding mirrors and a larger combiner screen would apper obtrusive and less elegant in a modern vehicle than simply using the windshield as a display surface.   .... '