Re Ambient Computing

Anytime, anywhere,  data and computation delivered.

Why the Future of the Computer Is Everywhere, All the Time

By The Wall Street Journal

October 31, 2022

In the ambient world, the technology is all around us, unseeable and untouchable.

Imagine this scenario in the not-too-distant future. You're awakened at 6:11 a.m. by the gentle sounds of tinkling bells and birdsong, even though you live in a 12th-floor apartment. Your alarm clock uses radar to track your breathing, and wakes you gently, with sound and light, when it detects you're in a lighter phase of sleep.

Your transition to wakefulness triggers a cascade of changes in your apartment. Your window shades open automatically. In the kitchen, coffee starts brewing. As you pad into the bathroom to brush your teeth, a display projected onto the mirror above the sink shows your calendar for the day. It highlights what time you'll have to leave to get to your office for the in-person meeting you scheduled for 8:30.

Returning to your bedroom, you find your stowaway robotic bed has retracted into the ceiling, and your collapsible walk-in closet has expanded to reveal your clothes and a full-length mirror. The mirror suggests, based on your schedule and the weather, an outfit it displays as an augmented-reality overlay that moves with your body as you inspect yourself. You aren't fond of the first option, so you make a swipe-left gesture in the air. The mirror responds by suggesting another outfit. You signal assent, and the two drawers containing the items you want glow around their edges, so you don't have to waste time hunting for them.

From The Wall Street Journal

View Full Article     

Google Talks Ambient Computing

New and old interfaces providing ambient computing

In AI.Googleblog 

Hidden Interfaces for Ambient Computing

Thursday, April 21, 2022

Posted by Alex Olwal, Research Scientist, Google Augmented Reality and Artem Dementyev, Hardware Engineer, Google Research

As consumer electronics and internet-connected appliances are becoming more common, homes are beginning to embrace various types of connected devices that offer functionality like music control, voice assistance, and home automation. A graceful integration of devices requires adaptation to existing aesthetics and user styles rather than simply adding screens, which can easily disrupt a visual space, especially when they become monolithic surfaces or black screens when powered down or not actively used. Thus there is an increasing desire to create connected ambient computing devices and appliances that can preserve the aesthetics of everyday materials, while providing on-demand access to interaction and digital displays.

Illustration of how hidden interfaces can appear and disappear in everyday surfaces, such as a mirror or the wood paneling of a home appliance.

In “Hidden Interfaces for Ambient Computing: Enabling Interaction in Everyday Materials through High-Brightness Visuals on Low-Cost Matrix Displays”, presented at ACM CHI 2022, we describe an interface technology that is designed to be embedded underneath materials and our vision of how such technology can co-exist with everyday materials and aesthetics. This technology makes it possible to have high-brightness, low-cost displays appear from underneath materials such as textile, wood veneer, acrylic or one-way mirrors, for on-demand touch-based interaction.

For insight into how this works, you can see this video about the Hidden Interfaces project

Hidden interface prototypes demonstrate bright and expressive rendering underneath everyday materials. From left to right: thermostat under textile, a scalable clock under wood veneer, and a caller ID display and a zooming countdown under mirrored surfaces.

Parallel Rendering: Boosting PMOLED Brightness for Ambient Computing

While many of today’s consumer devices employ active-matrix organic light-emitting diode (AMOLED) displays, their cost and manufacturing complexity is prohibitive for ambient computing. Yet other display technologies, such as E-ink and LCD, do not have sufficient brightness to penetrate materials.

To address this gap, we explore the potential of passive-matrix OLEDs (PMOLEDs), which are based on a simple design that significantly reduces cost and complexity. However, PMOLEDs typically use scanline rendering, where active display driver circuitry sequentially activates one row at a time, a process that limits display brightness and introduces flicker.

Instead, we propose a system that uses parallel rendering, where as many rows as possible are activated simultaneously in each operation by grouping rectilinear shapes of horizontal and vertical lines. For example, a square can be shown with just two operations, in contrast to traditional scanline rendering that needs as many operations as there are rows. With fewer operations, parallel rendering can output significantly more light in each instant to boost brightness and eliminate flicker. The technique is not strictly limited to lines and rectangles even if that is where we see the most dramatic performance increase. For example, one could add additional rendering steps for antialiasing (i.e., smoothing of) non-rectilinear content.

Illustration of scanline rendering (top) and parallel rendering (bottom) operations of an unfilled rectangle. Parallel rendering achieves bright, flicker-free graphics by simultaneously activating multiple rows.

Rendering User Interfaces and Text

We show that hidden interfaces can be used to create dynamic and expressive interactions. With a set of fundamental UI elements such as buttons, switches, sliders, and cursors, each interface can provide different basic controls, such as light switches, volume controls and thermostats. We created a scalable font (i.e., a set of numbers and letters) that is designed for efficient rendering in just a few operations. While we currently exclude letters “k, z, x” with their diagonal lines, they could be supported with additional operations. The per-frame-control of font properties coupled with the high frame rate of the display enables very fluid animations — this capability greatly expands the expressivity of the rectilinear graphics far beyond what is possible on fixed 7-segment LED displays.

In this work, we demonstrate various examples, such as a scalable clock, a caller ID display, a zooming countdown timer, and a music visualizer.

Realizing Hidden Interfaces with Interactive Hardware

To implement proof-of-concept hidden interfaces, we use a PMOLED display with 128×96 resolution that has all row and column drivers routed to a connector for direct access. We use a custom printed circuit board (PCB) with fourteen 16-channel digital-to-analog converters (DACs) to directly interface those 224 lines from a Raspberry Pi 3 A+. The touch interaction is enabled by a ring-shaped PCB surrounding the display with 12 electrodes arranged in arc segments.   .... ' 

Google Announces Ambient Mode on Some Phone Devices

What they have called an 'intent based' rather than 'App based' design, for some phones now, and likely to happen on all their devices. I would imagine this would work quite well, but better for some kinds of task-based proactive,  alert, need or goal approaches.   Might work well for team situations where there are shared goals and needs to coordinate efforts via an AI of some type.    This could mean that the device might proactively choose tasks or data to be shown, providing an element of direction and management?  I would consider the potential for workplace applications quite interesting, and even controversial.

" ... Ambient Mode brings a proactive Google Assistant experience to your Android phone. Arvind Chandrababu, product manager on the Assistant for Mobile team, shares how it easily shares the information that matters most to you.    https://www.youtube.com/watch?v=vaRzJo_19nY

From9to5 Google:

"... Ambient Mode lets you “turn your phone into a digital photo frame, control music, other smart home devices and more.” Enabled during charging, it is officially “available on select devices on Android O [8.0] and above. According to Ars Technica’s Ron Amadeo, devices from Sony and OEM Transsion will also receive Ambient Mode with wider availability next week. Lenovo already showed off two tablets with this experience. One way to think about this new mode is as a stock always-on display for OEMs that do not have their own custom implementations.  ... " 

Alexa Still Competing Best in the Home

Agree, have just been taking it into the car with Echo Auto.  Of course its still competing versus the smart phone screen.   In fact all connections with Echo devices are enabled and formulated from a smartphone App.   True you get higher audio and sometimes video from echo devices,  but not necessarily intelligence.   Siri and Android solutions can still be 'closer' to you because they are embedded in the hardware itself.   And hardware AI is starting to show its value for some kinds of AI problems.   The idea of very good 'ambient computing'  listening and engaging and conversing, may also be best closer to the hardware.   Good related thoughts below:

Alexa’s real competition is still your phone screen
Amazon is still better inside the home than outside
By Dieter Bohn in TheVerge

One of the big themes we’ve been tracking for a few years now is Amazon’s various attempts to make Alexa useful outside your home. Amazon has a very good value proposition for customers inside their houses: Echo speakers are great for music, timers and such in the kitchen, and smarthome controls.

Amazon clearly has ambitions to make Alexa the leading platform for ambient computing. But to do that, it needs more ubiquity than it can achieve right now. That’s one reason that Amazon was so excited to announce a partnership with GM to make Alexa available on those cars. It’s also one reason I was surprised to see the company didn’t announce any updates to the Echo Auto.

But the most obvious way to do that is to be the default assistant on phones. Amazon is probably never going to get there, because Apple won’t allow it, for one thing. On Android you can switch our your default assistant from Google to Alexa, but the number of customers who realize that’s possible is small and the number who are likely to do it is even smaller.  ... " 

Advancing AI in the Workplace: Ambient and Voice

Not un-expected, further intrigued about how much of it will be 'ambient', that is happening in the background and alerting the employee, and how much of it will be using voice control, after the very popular systems in the home.  Will Robo-Bosses become common?

Gartner: Get ready for more AI in the workplace in Computerworld
At the firm’s Digital Workplace Summit in London, analysts said they expect artificial intelligence to be common in the office by 2025; they already see ‘huge pent-up demand.’   By Matthew Finnegan

LONDON – Artificial intelligence (AI) will be widely adopted in office environments in a variety of ways over the next few years as businesses invest in digital workplace initiatives, Gartner analysts said today....

Ambient Computing on Alexa

Have not heard the term 'Ambient Computing' used too often, but is more general than voice.  Indicates a more conversational, but also notifying and two other way interaction.    I continue to like to add  the inclusion of the term 'Assistant', because it implies augmenting services. Here a good overview of the status of Amazon Alexa, with lots of links to related articles and resources. 

Amazon Alexa Enables an Ambient Computing Lifestyle  in InfoQ by Aslan Brooks

Alexa is becoming ubiquitous in the home through a purposeful Amazon Web Services (AWS) strategy to make Alexa, voice-activated service, a central component of an ambient computing lifestyle, according to Dave Limp, Amazon’s senior VP of devices and services in a recent article in the Verge.  AWS strategy has contributed significantly to the selling of 100 million devices with Alexa built-in.

Amazon first released Alexa, voice assistant, with the Echo and Echo Dot in 2014. Alexa listens for a magic wake-up word and then in real time records and sends your words to the AWS cloud for processing. Alexa uses automatic speech recognition (ASR) to transform words into text. The text gets processed by "skills" which are subscribed to by the user of Alexa. AWS has built a platform called Lex that Alexa uses for ASR and natural language understanding (NLU). AWS provides Lex to developers, so they can use it to build applications separate from Alexa.   .... " 

Note also the link to related science papers.

Microsoft and AI

During the last AI go around, we worked with Microsoft.  They were a natural partner, because they were also providing all the fundamental things to get work done.  And get it done more intelligently.   We were early experimenters with advisory systems like 'Paperclip', a form of Ambient computing.

Then they did seem to fade. And Apple, IBM, Google and even Amazon have taken over.    Is Microsoft coming back?  How and what parts should we pay attention to?   How should we use them? Integrate their methods with our own developments?   And use their data and communications integrations like Linkedin, Skype and Cortana?

Inside Microrosft's AI Comeback  Jessi Hempel in Wired:

" ... Yes, Microsoft. His bet is that the former kingdom of Windows alone has the capability to establish itself as AI’s third giant. It's a company that has the resources, the data, the talent, and—most critically—the vision and culture to not only realize the spoils of the science, but also push the field forward. In January, in a move noted throughout the industry, Bengio agreed to be a strategic advisor to the company. This gives Microsoft a direct line to one of AI’s top resources for ideas, talent, and direction. And it’s a strong sign that Microsoft actually has a shot at making the ruling AI duo into a trio. ... " 

Apple Announces Home Assistant and Speaker: HomePod

Reportedly available in December at an expensive $349.    Featuring Siri for intelligence.

Apple's making an Amazon Echo competitor called 'HomePod'   by Ben Gilbert

Apple is making a competitor to Amazon's Echo speaker, and it features a familiar voice: Siri. Apple's new device is called HomePod, and it marries Apple's Siri digital assistant to a home speaker.... "

Based on what was said at today's WWDC it looks like Siri could be a formidable delivery mechanism for intelligence.   Also impressed by their voice synthesis advances to give the device character.  Will that impress customers enough to buy?  Where does intelligence combine with facility to produce value I really want?

(Update)    An interesting point. Apple does not believe in Ambient intelligence?
Apple Is Countering Amazon and Google with a Siri-Enabled Speaker
Oddly, though, Apple is emphasizing HomePod’s musical skills rather than all-around smarts.
by Rachel Metz  ...  In Technology Review

More on my look at many Assistants 

(Update) Essential Home as Inter-Operable Ambient OS

I like the term Ambient Computing or Ambient OS.   It listens, answers, performs, adapts.  A virtual assistant too.  A more private, more home oriented device?  Centralizing smart home actions.  IFTTT like?   Display too.  Not available or priced yet.

This is the new Ambient OS, Essential’s big bet to control your home
If this, then that, but much bigger ....  by Nilay Patel

It’s a huge morning of news from Andy Rubin’s Essential: a new Android phone with a unique edge-to-edge display, a modular accessory system, a new 360 camera for that system, and a new intelligent speaker with a huge circular display.

But the biggest news of all might be the OS that runs on that speaker. Essential is calling it Ambient OS, and says that the goal is to “activate” your home by understanding the physical layout of your home, its occupants, and the various services and devices available to them. Ambient calls this “activating” your home, and says the Ambient OS is an “API” to all those things in combinations, allowing new kinds of applications to exist on top of them — like flashing your lights when a timer goes off. .... " 

(Update)   Listened to the Andy Rubin's Recode interview on this.  The surprising thing about the Essential Home device, driven by the Ambient OS,  was that it is made to be inter-operable with other assistants, while also having its own.   So it is likely to work with Alexa, which is well known to be open to be integrated in hardware other than Amazon's.  Google Home also likely.  Unclear if Siri or Cortana would participate.    Still no indication of cost or when it might be available ....

Part of this talk was about the Essential Phone, which will be released in 30 days.  It was mentioned that the phone is not required for the Essential Home.  Not clear if some App in some phone is required for set up. ....

Processor Runs from Ambient Radio Waves

In Extremetech: Consider the Internet of Things or Wearable implications

" .. A new research initiative between a University of Washington team and TU-Delft in the Netherlands has created a microprocessor that can power itself through stray radio waves and receive programmable updates in the same fashion. While the CPU in question is exceedingly weak by modern standards (it’s a RISC-derived 16-bit microcontroller clocked at just 16MHz), it’s an order of magnitude more powerful than any other device that’s powered by ambient energy in the environment with no battery required. This has significant implications for Internet of Things development and for ambient computing as a whole. ... ".