Hololens Use Example in Healthcare

 An example that as recently described to me in more detail. 

HoloLens Headsets Connect Patients, Metropolitan Doctors

By ABC News (Australia), October 31, 2022

HoloLens allows a nurse to pinpoint areas of concern to a specialist.

Microsoft HoloLens mixed-reality holographic headsets are connecting remote patients in Australia with metropolitan physicians.

The Bairnsdale Regional Health Service tested the technology in the Victorian town of Dargo, linking patients to specialists in Melbourne via a holographic consulting room.  Nurses don the headsets and are directed by doctors to examine patients, who can evaluate them with greater thoroughness than they can in standard telehealth sessions.

Said Bairnsdale Regional Health Service's Robyn Hales, "If we can get patients that utilize [HoloLens] on a really regular basis, decrease people's travel, decrease ambulances needing to go out and collect patients, that provides really timely care, which is a significant cost saver."

From ABC News (Australia)     

View Full Article

Microsoft Gets Army Augmented-Reality Headset Contract

Such uses could lead the way to business application, still uncommon.

Microsoft wins U.S. Army contract for augmented-reality headsets, worth up to $21.9 billion over 10 years   By Jordan Novet  

Microsoft will deliver to the U.S. Army over 120,000 devices based on its HoloLens augmented-reality headset.

The deal, which could be worth as much as $21.88 billion over 10 years, follows a contract Microsoft received to build prototype headsets for the U.S. Army.

The contract comes a year and a half after Microsoft won a cloud contract from the Pentagon that could be worth up to $10 billion.  ... "

HoloLens2 Available and Shipping Now

Considerable overview piece from Microsoft.  They are now hooking it to a number of capabilities such as Dynamics and Azure driven Edge Computing.    Impressive. If significant mixed reality applications are out there, they could well be the leaders.

The making of the HoloLens 2: How advanced AI built Microsoft’s vision for ubiquitous computing  By John Roach

Video demonstrations at the link.
Business apps available now

Pilot and deploy quickly with a full suite of Dynamics 365 mixed reality applications  
Create spatially aware holograms.  Use the world as your canvas
Build cross-platform, spatially aware mixed reality experiences.
HoloLens 2   Learn more about the ultimate Intelligent edge device.
Azure   Explore services and devices for mixed reality development.
Dynamics 365 Get started on day one with business-ready applications.

Redmond, Washington – The first time people don the new HoloLens 2 on their heads, the device automatically gets to know them: It measures everything from the precise shape of their hands to the exact distance between their eyes.

The artificial intelligence research and development that enabled those capabilities “was astonishingly complicated” but essential to making the experience of using the device “instinctual,” said Jamie Shotton, a partner scientist who leads the HoloLens science team in Cambridge, United Kingdom.

“We want you to know how to use HoloLens without having to be taught how to use it,” he said. “We know how to interact with things in the real, physical world: We pick things up, we press buttons, we point to things. We aim, as far as possible, to translate that directly into mixed reality.”

Microsoft today announced the HoloLens 2 is now shipping to customers. The sensor-packed holographic computing headset uses AI to displace space and time, creating a mixed reality of people, places and things in order to facilitate one of our most basic human impulses: exchanging knowledge.

Microsoft Technical Fellow Alex Kipman said the headset defines the highest watermark of intelligent edge devices – AI-capable technologies that can collect and process data even without a reliable internet connection – and that can then share some or all of that data with the intelligent cloud when connected.

On a recent day, Kipman sketched a diagram of this ubiquitous computing fabric on a digital whiteboard in his office.

“HoloLens,” he said, “is the first native device to be invented from the ground up with this worldview in mind.”  .... "

Microsoft: How Hololens is used by Airbus

Good example of the use of Mixed/Augmented reality approaches in heavy industry.  Expect to see more of these to emerge. 

https://www.youtube.com/watch?v=lxjC4Z05qh8  Video overview of use

Airbus drives innovation and accelerates production with Azure mixed reality and HoloLens 2

 Microsoft HoloLens

Airbus—a pioneer in aerospace technology and leading designer and manufacturer of commercial and military aircraft, satellites, and launch vehicles—has made a commitment to transform traditional industrial processes through use of mixed reality. The company has partnered with Microsoft to use Azure mixed reality and HoloLens 2 as a way to accelerate the design and manufacture of aircraft, while increasing safety and functionality and ensuring professional development opportunities for employees. Intelligent services like Azure Remote Rendering are changing the way complex ideas are communicated, and the ability to build multi-user, spatially aware applications with Azure Spatial Anchors promotes greater collaboration while increasing quality, safety, and security. Airbus is using Azure mixed reality to unlock the full potential of HoloLens 2 and, as a result, expects to reduce design validation time by 80 percent and accelerate complex tasks during assembly by 30 percent.  

Learn more: 
Azure Mixed Reality:    https://azure.microsoft.com/en-us/topic/mixed-reality/
HoloLens 2: https://www.microsoft.com/en-us/hololens 
Azure Cognitive Services:  https://azure.microsoft.com/en-us/services/cognitive-services/

Airbus Tests Hololens

An example of the use of Hololens and VR/Mixed Reality for advanced manufacturing systems.  Most obvious space for this, which we tested .... integrating large quantities of background information such as blueprints and online analyses and metadata for manufacturing compliance specifications. 

Airbus works with Microsoft to trial HoloLens in aircraft manufacturing
Aircraft company Airbus believes mixed reality systems, powered using headsets such as HoloLens, will drive its digital manufacturing strategy .... "

Augmented Reality Adoption in Retail

I think this still has some way to go, especially with regard to getting consumers comfortable with the idea of using AR publicly.   That is why some wearable like smart glasses may be key to making that happen.    Even Microsoft with its HoloLens has admitted this by for now targeting the enterprise rather than consumer with their AR offering.    When with average consumer in average contexts be ready?

Will retailers be ready when AR adoption takes hold?    by Nikki Baird in Retailwire with Expert comment on the link

Through a special arrangement, presented here for discussion is a summary of a current article from the blog of Nikki Baird, VP of retail innovation at Aptos. The article first appeared on Forbes.com.

If consumers want to shop and pay on Instagram or use Facetime to co-shop with distant friends, retailers better figure it out. That challenge is no different when it comes to augmented reality (AR).

The challenge for retailers, who operate thin-margin businesses and don’t have a lot of extra cash available to experiment, is trying to decide when the time is right to “get in” on a new technology.... "

A Look at Future HoloLens Mixed Reality

Good half hour Podcast interview about HoloLens-2 current and future on Twit-TV

Gives a good update of HoloLens at Microsoft and how they are now focusing its uses

Today's special guest engineering GM for mixed reality apps, Lorraine Bardeen of Microsoft.  Lets talk more about what MS is doing with HoloLens, iOS, Android in mixed reality. ... 

Links to Dynamics 365 Mixed Reality and Perception Apps

Use for First line workers.  Field service inspection.

Initially will be partially delivered on IOS, Android phones. Then to applications with a HoloLens.

Now looking directly at manufacturing and supporting live demonstration, inspection, maintenance.

Apparently are not driving at a consumer orientation at this time.

This is exactly what we saw in the 80s for plant maintenance, and in some much rarer areas for previewing marketing in place.

HoloLens 2 Open for Preorder

Early reviews seem to say it is a considerable improvement.    Targeted towards business rather than games.  Still quite expensive for the consumer,  $3500 per device.   How much directly usable off the shelf applications are available?   Lots of demo videos below.  Interest in preorder, with no date of shipment mentioned?

HoloLens 2
Mixed reality is ready for business
A new vision for computing, built on a history of innovation

Mixed reality on HoloLens 2 combines an untethered device with apps and solutions that help people across your business learn, communicate, and collaborate more effectively. It's the culmination of breakthroughs in hardware design, artificial intelligence (AI), and mixed reality development from Microsoft, designed to help you lead your industry into the future—starting today.  .... "

Good podcast at Twit: Windows Weekly overview:   https://twit.tv/shows/windows-weekly/episodes/610?autostart=false 

Hololens 2 to have Eye Movement Detection

Good addition of detecting and I assume controlling with eye movement.   The control by gesture has never been good enough for many applications.  Need more general applications to get people used to the interaction

Microsoft HoloLens 2 augmented reality headset unveiled  By Leo Kelion in BBC Tech.

Microsoft has a new version of its augmented reality headset, which now detects where its users are looking and tracks the movements of their hands.

It said that HoloLens 2 wearers would find it easier to touch and otherwise interact with graphics superimposed over their real-world views.

Other improvements include filling more of a user's view and automatically recognising who they are.

The firm is pitching the kit as being ready for use in business environments.

Many experts believe mixing together graphics and real-world views has greater potential than virtual reality, which removes the user from their immediate environment. .... "

Holoforge for Historical Experience

Saw a number of examples early on of using VR to effectively show historical data,  good example for use, for both teaching and for interacting with complex data.

The Mont Saint-Michel: Digital perspectives on the model
Celebrating French art, history, and culture through mixed reality and artificial intelligence.
Share

The Musée des Plans-Reliefs is bringing to life the historic Mont-Saint-Michel relief map—an example of the 17th century’s most advanced mapping technology—via a mixed reality experience that uses current mapping innovations to immerse viewers in a vital piece of French history and culture.

For centuries, technology has been influencing the way people engage with the world and shape the course of history. In 17th and 18th-century France, large-scale 3D maps—painstakingly built by hand down to the most intricate details—were the most advanced mapping technology of their time. They were considered such valuable strategic tools that leaders like Napoleon and King Louis XIV considered them military secrets and hid them from public view.

The Musee des Plans-Reliefs in Paris is home to more than 100 of these historic relief maps that  have withstood the test of time. But the crown jewel of the collection is the model of Mont-Saint-Michel—a rocky headland off the Normandy coast with a Benedictine abbey that’s an architectural marvel in its own right—presented by a monk to Louis XIV in 1701. A new HoloLens experience is bringing these technological feats, both the relief map and Mont-Saint-Michel itself, to life for a new generation.

The museum is partnering with Microsoft, HoloForge, and Iconem to create “The Mont Saint-Michel: Digital perspectives on the model,” a HoloLens experience celebrating French culture and innovation. The goal of the exhibit is to use mixed reality technology in a way that empowers the Musée des Plans-Reliefs to unlock a more vital kind of storytelling.  ....  " 

NASA and Other Users of Hololens

Handsfree with reference to documentation and process.    Similar to our experiments.

NASA is using HoloLens AR headsets to build its new spacecraft faster

Lockheed Martin engineers wear the goggles to help them assemble the crew capsule Orion—without having to read thousands of pages of paper instructions.
by Erin Winick  in Technology Review

When you work at a factory that pumps out thousands of a single item, like iPhones or shoes, you quickly become an expert in the assembly process. But when you are making something like a spacecraft, that comfort level doesn’t come quite so easily.

“Just about every time, we are building something for the first time,” says Brian O’Connor, the vice president of manufacturing at Lockheed Martin.

Traditionally, aerospace organizations have replied upon thousand-page paper manuals to relay instructions to their workers. In recent years, firms like Boeing and Airbus have started experimenting with augmented reality, but it’s rarely progressed beyond the testing phase. At Lockheed, at least, that’s changing. The firm’s employees are now using AR to do their jobs every single day.  ... "

Microsoft Pushes Big Changes in AI

Missed this, again the emphasis is towards business solutions.  Have not seen much from Cortana being useful for consumers.  Have tested it for business, where it should have value, and quickly abandoned it for regular use.

Are we in the A.I. revolution? Microsoft’s keynote points to huge change   By Kevin Parrish in DigitalTrends

Microsoft’s annual Build conference for developers kicked off the week starting with a keynote provided by CEO Satya Nadella. He didn’t reveal any new Windows 10 hardware, but instead focused his presentation on solutions spanning from the intelligent cloud to the intelligent edge, the latter of which simply classifies everything we use ranging from smart thermostats to HoloLens in the corporate environment.

He began the keynote emphasizing how artificial intelligence is changing the world, drawing parallels to the Industrial Revolution where, in a span of 20 years, the nation moved from horse-drawn engines to automobiles. The core technologies of the Industrial Revolution were essentially invisible: the combustion engines in cars, plumbing in buildings, the installment on electrical systems for lighting, and so on. Artificial intelligence and the cloud are just as revolutionary and invisible.

One example is Microsoft’s new partnership with civilian drone manufacturer DJI. Commercial drones will feature an integrated Azure IoT Edge component, enabling customers to deploy an artificial intelligence model developed in the cloud right to the drone itself in real time. Even more, that A.I. model can be deployed across the corporation’s drone army scattered across the globe in mere minutes.

In an on-stage demo, Azure Internet of Things director Sam George showcased an Azure IoT Edge-based smart camera produced by Qualcomm that detected a stress fracture stemming from a tank within a facility. A second camera discovered a pressure drop on the roof. That is where DJI’s drone comes in: Instead of sending a technician on a long, dangerous journey up to the roof, a company can simply send a smart drone to investigate and correctly identify the problem. Moreover, the drone can use the same A.I. model downloaded to Qualcomm’s camera. This advancement can save both time and money while keeping workers safe. ... " 

Remote Vehicle Interaction

Beyond the Driverless Car.

Team of Engineering, CS Students Enhance Future of Remote Vehicles 
Embry-Riddle Aeronautical University
By Katharina Lane

Embry-Riddle Aeronautical University researchers have combined a small unmanned aircraft system (sUAS), an augmented reality (AR) headset, and a neuromuscular gesture recognition armband to develop a tool that enables unmanned aircraft pilots to safely operate remote aerial vehicles while remaining mobile and carrying out other tasks. The new system is a wearable technology that mimics natural human-to-human interaction. In addition, the AR system lets users communicate with each other, and can be programmed to enable users to access only information they have clearance to see, says Embry-Riddle's Adam Berlier. He notes the system relies on a Microsoft HoloLens AR headset and an MYO gesture recognition armband (previously developed by Embry-Riddle researchers) so users can retain situational awareness while controlling the remote vehicle. "In addition to first responders, this technology would be very useful for sUAS inspections of structures such as power lines or wind turbines," says Embry-Riddle's Brandon Koury .... "

Cheaper Chrome Devices

Was reminded once again of the $100 laptop push of a decade ago that was aimed at schools.  Chrome Laptops are a natural next step, with much of the function needed for productivity online.   And, of course,  Microsoft plans to add cheaper mixed reality through Hololens.

Windows 10 laptops for schools. And a big education push for 2018
By Tom Warren @tomwarren in TheVerge ... "

Windows Insider on Mixed Reality

Been following Microsoft's interest in 'mixed reality' aka augmented reality for a client.  Just discovered Microsoft insider podcasts, which looks at near future directions in Windows operating systems,  applications and their potential use in the enterprise.  Most recently they have just featured a considerable podcast on mixed reality that is worth a look.  Quote:

Alex Klipman:  (from video):  Now we're standing together at the threshold of the next revolution of computing.  Now, the thing that excites me about this revolution is that computers will empower us to renegotiate our very contract with reality, giving us the capability to transcend time, space, and devices

In this revolution we will immerse ourselves in virtual worlds of our choosing, and we'll be able to accomplish impossible things.  And we'll be able to do all of this while creating lasting memories with the people that we love

Our very sense of reality is set to be transformed as we enter this new era of computing, the era of mixed reality .... " 

Update on Hololens

Micorosft's update on  AR work.  Always looking for practical business examples.   A bit thin on new applications covered.  Here covered:

Driving collaboration in car design
Ford design and engineering teams are able to iterate more quickly when they use HoloLens to visualize full-scale models in 3D. Processes that used to take weeks now take days, because they no longer need to physically build every prototype. ... 

Modernizing crime scene investigation
HoloLens enables police officers to capture evidence in real time with tuServ, Black Marble’s award-winning app for police operations. Gathering digital evidence decreases the risk of disturbing physical evidence. .... 

Bring life to Mixed Reality
Create holograms of people and performances for your mixed reality, virtual reality, and augmented reality experiences. Mixed Reality Capture Studios are now open San Francisco, Seattle, and London. ... "

Ford Using MS HoloLens for Car Design

Complex and cooperative design continues to be a place where augmented reality is likely to be valuable.

Ford Using Microsoft HoloLens For Car Design By Rebecca Hills-Duty in VR Focus.

HoloLens allows designers and engineers to prototype new cars faster and more cheaply.
Several businesses have been exploring the possibilities offered by Microsoft’s mixed reality (MR) headset. Several car manufacturers such as Volvo and Audi have been experimenting with augmented reality (AR) and MR, and now Ford is joining them.

Ford were already using the HoloLens in a limited capacity, but now the company is expanding its use after a pilot phase. Usually when designing a car, designers and artists will often start with simple pen and paper, then move on to crafting a solid 3D version using modelling clay. Designers and artists can now use HoloLens to adjust 3D models, taking into account real-world limitations and adjustments without needing to rebuild the model from scratch every time.

Designers and engineers can put an overlay over a currently existing car and prototype adjustments for the new model without needing to completing built it anew, or physically build every new prototype. Designers and engineers save a great deal of time and money be using this approach with the HoloLens. It also means that radical new ideas and concepts can be considered, and many more ideas tried out to find the perfect fit. .... " 

Augmented Reality Challenge

A look at Hololens vs ARKit

Is Augmented Reality going to be Microsoft’s next smartphone moment?
HoloLens was inspirational, but it’s ARKit and ARCore that are going to win developers.  By Peter Bright in ARsTechnica

Mixed Reality Partner Program

Received:

Introducing the Mixed Reality Partner Program

If your company is a digital agency, systems integrator, or solution provider, the Mixed Reality Partner Program can equip your company with the skills to successfully drive commercially-focused mixed reality solutions for your clients. .... " 

HoloLens 2 Adding AI Processor

Augmented realities will need better edge intelligence to provide value.   By John Brandon, Computerworld.  Some interesting applications are described.

Why Microsoft is adding an AI co-processor to HoloLens 2

With more processing power for AI, the HoloLens 2 could help us better interpret data and make better business decisions ... ' 

" ... Microsoft knows this, and that’s why a recent announcement about adding an AI co-processor to the HoloLens 2 caught my eye. The chip will help with tagging data in the real world, which sounds just as complex and compute intensive as you can imagine. Also, this processing will work much faster when the chip runs locally instead of in the cloud ...  "