A Wearable Robotic Assistant That's All Over You

 A Wearable Robotic Assistant That's All Over You

IEEE Spectrum

Mat Sadowski, June 19, 2023

The Calico assistive robot developed by researchers at the University of Maryland can be worn on the user's clothes, where it runs along a track to perform various tasks. The 18-gram robot can localize itself and map a path with sensors that perceive magnetic neodymium markers embedded in the wearer's apparel. Calico can bear a 20-gram payload and travel between 15 millimeters (0.59 inches) and 227 millimeters (8.93 inches) per second, powered by a battery that holds more than eight hours' charge when idle and 30 minutes' charge when moving. The device can show users its progress on tasks by transforming their arm into a physical progress bar, moving further up the limb as they reach daily goals. .. ' 

Google Tests AI Robotics in a Kitchen

Look forward to seeing this.  Kitchen has many tasks that could be automated, but usually not well positioned for broad application and integration.  Note here a number of hints at Google investment in the topic.  Ready to test.

Google Is Testing Its Latest AI-Powered Robot In a Kitchen  in ExtremeTech   By Adrianna Nine on August 17, 2022

Many robots are created to conduct highly-controlled jobs like frying potatoes, watering plants, or collecting litter. But a truly life-changing robot is one that can adapt to changing—and sometimes hectic—circumstances (ideally without losing its cool). That’s the line of thinking Google is following as it meshes its language-handling AI with a handy robot assistant.

Google’s Pathways Language Model (PaLM) is a relatively new 540-billion parameter network built to complete a variety of complex language-based tasks. It’s said to be intelligent enough to describe how it solved a math problem and annoy you by explaining your own jokes. Rather than focusing on one area of “expertise” and starting fresh every time it learns a new skill, PaLM can “stack” previously-learned knowledge to devise solutions to novel problems, similar to how humans assess new situations. This is important if a robot is meant to help humans in their jobs or day-to-day personal lives.

It just so happens that Google’s parent company, Alphabet, has been working on a new robotics firm called Everyday Robots. As its name suggests, the firm’s goal is to build robots that learn on their own and take care of “time-consuming, everyday tasks.” Combined with PaLM, Everyday Robots’ SayCan robot becomes the PaLM-SayCan, a bot capable of assessing its own capabilities, its environment, and the task at hand, then breaking that task into smaller sub-tasks to achieve the desired goal. ... ' 

Rephrasing Communications in Context

 Advances in assistant communications.  In Google Blog,  As always context is very important. 

Contextual Rephrasing in Google Assistant

Tuesday, May 17, 2022

Posted by Aurelien Boffy, Senior Staff Software Engineer, and Roberto Pieraccini, Engineering Director, Google Assistant

When people converse with one another, context and references play a critical role in driving their conversation more efficiently. For instance, if one asks the question “Who wrote Romeo and Juliet?” and, after receiving an answer, asks “Where was he born?”, it is clear that ‘he’ is referring to William Shakespeare without the need to explicitly mention him. Or if someone mentions “python” in a sentence, one can use the context from the conversation to determine whether they are referring to a type of snake or a computer language. If a virtual assistant cannot robustly handle context and references, users would be required to adapt to the limitation of the technology by repeating previously shared contextual information in their follow-up queries to ensure that the assistant understands their requests and can provide relevant answers.

In this post, we present a technology currently deployed on Google Assistant that allows users to speak in a natural manner when referencing context that was defined in previous queries and answers. The technology, based on the latest machine learning (ML) advances, rephrases a user’s follow-up query to explicitly mention the missing contextual information, thus enabling it to be answered as a stand-alone query. While Assistant considers many types of context for interpreting the user input, in this post we are focusing on short-term conversation history.

Context Handling by Rephrasing

One of the approaches taken by Assistant to understand contextual queries is to detect if an input utterance is referring to previous context and then rephrase it internally to explicitly include the missing information. Following on from the previous example in which the user asked who wrote Romeo and Juliet, one may ask follow-up questions like “When?”. Assistant recognizes that this question is referring to both the subject (Romeo and Juliet) and answer from the previous query (William Shakespeare) and can rephrase “When?” to “When did William Shakespeare write Romeo and Juliet?”

While there are other ways to handle context, for instance, by applying rules directly to symbolic representations of the meaning of queries, like intents and arguments, the advantage of the rephrasing approach is that it operates horizontally at the string level across any query answering, parsing, or action fulfillment module.  ... '

A New Way of Assistant Activation

Works well if  you are by the device.   Voice too?  Makes sense.

Google Assistant ‘Look and Talk’ Activation Heads Toward Release

ERIC HAL SCHWARTZ on April 14, 2022 at 4:00 pm

Google is beta testing a way to awaken Google Assistant with a glance instead of a spoken wake word on the Nest Hub Max, according to a report from 95o5 Google. Despite early rumors about ultrasound sensors, the “Look and Talk” feature will use the Google Nest Hub Max smart display’s camera to spot and activate Google Assistant when a user is looking at the camera built into the screen.

LOOK AND TALK  ... 

The facial recognition wake word appears to be in beta testing right now, without any formal announcement about when it will become available. Look and See is a new name for what was previously called Blue Steel,  a reference to the signature facial expression of Ben Stiller’s character in the film Zoolander. Turning on Look and Talk would make the Nest Hub Max’s camera a visual version of the “always listening” aspect of Google Assistant. Instead of saying “Hey, Google,” and issuing voice commands, users could simply look into the camera from up to five feet away and just start making requests of the voice assistant.   .... ' 

Voice Assistant use on Smart Phones

 Less use? Is it a matter of context and environment, or capability? 

Smartphone Voice Assistant Use Stalls Out But Consumers Want More Voice Features in Mobile Apps – New Report

BRET KINSELLA on February 9, 2022 at 4:51 pm   in Voicebot.ai

The use of general purpose voice assistants such as Siri, Alexa, and Google Assistant contracted over the past 12 months according to data in a new report by Voicebot Research. A national consumer survey of over 1,100 found that fewer U.S. adults said they were using their smartphone-based voice assistant while monthly active users (MAU) also declined as a percent of the population   ... ' 

Learn to Create Voice-Plus-App Experiences, with No Coding Required

Amazon-Alexa continues to push Voice-App experiences: 

Learn to Create Voice-Plus-App Experiences, with No Coding Required

Entice Alexa customers to go deeper with your brand by extending into your apps and websites. Don’t miss next week’s Alexa Tech Talk to learn how to create compelling voice-plus-app experiences with Alexa for Apps.

Title: How to Build with Alexa for Apps

Date: Wednesday, Jan 19, 2022

Time: 10:00 am - 10:30 am PST

Register now ... 

Amazon Does a Home Robot: Astro

A number of such 'friendly' ranging home robots have been around, none successful.  I recall the Pepper, Yumi, Kuri, Jubi, Aibo robots.  All mentioned here.  Eldercare being hinted at.  Has also been rumored to be under development for years.  This is expensive. Notably it will not climb stairs, or I would assume even a single step, or rugs?

Amazon Does Astro,  a Rolling Home Robot

Introducing Amazon Astro, Household Robot for Home Monitoring, with Alexa, Includes 6-month Free Trial of Ring Protect Pro

Brand: Amazon  Day 1 Editions

Price: $999.99 & FREE Returns, or 5 monthly payments of $200.00

Introductory price. After introductory period, price will be $1,449.99. Terms and conditions apply., Available exclusively by invitation, Ships from and sold by Amazon.com Services LLC.,Please note this product can only ship to addresses in the 50 US states.

Keep home closer - Meet Astro, the household robot for home monitoring, with Alexa.

Introducing Intelligent Motion - Amazon Astro uses advanced navigation technology to find its way around your home and go where you need it. When you're not using Astro, it will hang out close by at the ready.

Stay connected from anywhere - Remotely send Astro to check on specific rooms, people, or things. Plus, get alerts if Astro detects an unrecognized person or certain sounds when you're away.

Unlock even more peace of mind - Activate your 6-month free trial of Ring Protect Pro subscription and have Astro proactively patrol, investigate activity, save videos in Ring's cloud storage for up to 60 days, and more.

Alexa Together subscription (Coming soon) - Remotely care for aging loved ones, giving you peace of mind while helping them live independently. Set up reminders, manage shopping lists, receive activity alerts, and more.

Put Alexa in motion - Astro can follow you with entertainment or find you to deliver calls, messages, timers, alarms, or reminders.

Designed to protect your privacy - Turn off mics, cameras, and motion with one press of a button and use the Astro app to set out of bounds zones to let Astro know where it's not allowed to go.

Customize with compatible products – Astro comes with a detachable cup holder and can carry other items (sold separately) like a Ziploc container, the OMRON blood pressure monitor, and a Furbo Dog Camera that tosses treats to your pet.

We want you to know:  Astro cannot go up or down stairs.

Advances in Baidu Brain

Baidu is coming back, had looked at some of their earlier offerings and was not impressed, I get the impression they are now more seriously looking at the assistant market.  

Baidu Unveils Upgraded Baidu Brain AI and Smart Devices  By ERIC HAL SCHWARTZ 

Chinese tech giant Baidu showcased its latest creations at its yearly Baidu World conference. Much of the broad array of new products are powered by the new and improved Baidu Brain AI platform revealed at the same time, including robotic vehicles and the first smart television controlled with the Xiaodu voice assistant.  ... 

Baidu’s centerpiece of the event was its new robotic car, built with Baidu’s Apollo AI driving program. Apollo is embedded in the vehicle and is able to improve how well it performs not only in driving but in communications and interactions with passengers. Baidu has plans for automated taxis and other autonomous vehicles and is at the point of demonstrating them at a larger scale than most of its rivals. On the more domestic front, Baidu’s Xiaodu voice assistant has a few new homes, including its first smart television. The 86-inch TV is built with as much advanced tech as could be crammed within, including Xiaodu running the whole thing through voice and gesture commands. Xiaodu is also popping up in a new smart display and active noise-canceling headphones.  ... '

Alexa now with Alternative Phase Triggers

Interesting, but still very simplistic.   I still have to know the exact phrase to make it work.   Now have had Alexa in place for years, and I still have to do trial and error for many interactions. She has trained me for many, but not all interactions.    I still like the ultimate assistant concept being demonstrated in front of me or my guests.  But its not here yet. Its Harder than we thought 

Alexa Routines Add Alternative Phrase Triggers

By Eric Hal Schwartz  on July 28, 2021 at 1:00 pm

Amazon has augmented Alexa routines with a more open approach to activation commands. Users creating custom routines initiated by a specific phrase can now add additional terms to begin the routine.

Flexible Talk

Alexa routines are a set of interrelated actions taken by the voice assistant and connected smart devices all triggered by device owners can set up routines on the Alexa app to play out a series of actions based on a voice command, alarm or another trigger. Though their scope has grown enormously, each custom routine was connected to a single activation command. Now, people creating custom routines can connect them to as many as seven phrases. For instance, saying “Alexa, good morning,” “Alexa, the sun’s up,” or “Alexa, breakfast time,” could all start a regular set of activities for Alexa when the user wakes up.   ... '

Amazon Alexa Live 2021

Attended some technical portions of this, was informative. Here a followup of things presented, announced.

Everything new announced at Amazon Alexa Live 2021

By Michael Bizzaco, July 21, 2021 9:00AM

Amazon Alexa is one of today’s go-to voice assistants. Available on hundreds of devices, from smart speakers to displays and thermostats, Alexa grows more popular every day, with over 100,000 million device owners and 900,000 registered developers producing Alexa-powered products. Speaking of the latter, Alexa Live 2021 has finally arrived. This year’s free virtual symposium is a great place to learn about all of the new developer tools and services that Alexa will be capable of in the near future. Here’s everything announced at this year’s event.   ... '  

Wheelchairs with Smarter Voice Controls

Seems a very natural solution.  Especially for integrating with chair and smartphone and other aids. Incudes data important to data and operator.

Permobil Connects Electric Wheelchair App to Alexa and Google Assistant   By Eric Hal Schwartz

Electric wheelchair manufacturer Permobil has released voice apps connecting Google Assistant and Amazon Alexa to its MyPermobil mobile app. Permobil owners can now get access to the current status of their wheelchair with voice commands.

WHEELED VOICE

The MyPermobil app collects real-time information about the wheelchair’s battery charge, how far it can travel before running out, and the distance it has already traveled since it was last plugged in. The app also gathers a report on the previous week and month’s use of the power seating feature of Permobil wheelchairs, which adjusts the position of the seat and legs to prevent the person in the wheelchair from being in one position for too long. Now, Permobil owners can now connect the app to Google Assistant or Alexa and ask the voice assistants to pull up all of that information without needing to take out their smartphone. Plenty of wheelchair users are not able to reach or control their smartphones very easily or at all. The option to interact with the app using voice commands is their first opportunity to access crucial information about their wheelchair.  ... "

Samsung Looks at an Assistant Again

 We tested Samsung's Bixby assistant offering for product interaction tests.     Now they appear to be replacing it with a new or updated approach.   With hint of an interface in 3D.

Samsung May Replace Bixby With a 3D Virtual Assistant Named Sam  in Voicebot.ai

Samsung may be creating a new virtual assistant named Sam that may accompany, or possibly replace, ith a 3D Virtual Assistant Named Sam

ERIC HAL SCHWARTZ  Bixby. As first discovered on Twitter, Lightfarm Studios and the Samsung-owned Cheil Agency showcased the three-dimensional character in a since-deleted webpage available only through the Wayback Machine.  ..." 

Smart Speaker Market

I  consider the smart speaker phenomenon a means of testing how we can interact with computation, related intelligence and useful .   So the measure seen here is how well we are doing to date.  Still behind I believe,  but we are continuing to learn.

U.S. Smart Display User Base Grew by More Than 50% in 2020   By Bret Kinsella in Voicebot.ai

While the smart speaker market’s U.S. adoption rate barely grew in 2020, smart displays followed another path entirely. Voicebot has tracked smart display adoption multiple times per year since 2018 and 2020 witnessed a surge in device adoption. Just over 16% of U.S. adults that owned smart speakers at the beginning of 2020 had at least one smart display in their device collection. By September, that figure reached 24.1% and it was 25.8% in January 2021. That is a significant rise considering the previous 12-months saw only about a three percentage point rise. The 2020 growth rate was three times higher.

This trend data were first reported in Voicebot’s U.S. Smart Speaker Consumer Adoption Report 2021. The report includes over 30 pages of analysis, hundreds of individual data points, and 35 charts, including several that address smart speaker and smart display market share by vendor and installed user base.

VIDEO CHAT IS THE DRIVER

It may seem counterintuitive that smart displays grew so quickly and yet the overall smart speaker market did not. The answer is embedded in the chart above. New smart display owners in 2020 were not new smart speaker owners. Instead, they were existing smart speaker owners that were acquiring their first smart displays during the global COVID-19 pandemic. So, they didn’t expand the smart speaker user base very much but they did add a new voice interactive device type to many consumer homes.  ... '

Augmented Reality Remote Assistants

 Had not thought of this specifically, ot a bad idea if it can be done well.  Needs to be taken beyond voice.   But what takes it most effectively beyond the screen?   Does now ubiquitous conference call show the way?

Augmented Reality Remote Assistant: Changing the World   By Vinod Janapala in CustomerThink

2020 was a turning point when part of the usual life went online. Therefore, digital solutions for different spheres of life began to develop rapidly. Now almost every company switch to digital transformation to stay in tune with present-day realities: cloud technologies, zoom conferences, and deep learning algorithms.

Augmented reality is an essential technology for players of any sphere. AR is used as a way to remotely process and control data. One of the modern solutions is an AR-based remote assistant.

About the AR Remote Assistant

It is a new technology that allows the creation of guidelines for the second party via the Internet. Initially, there was a tech solution in which an expert could use the chain of two technical devices for temporary viewing via a network. The purpose was to fix bugs in software.

The innovative AR remote assistant combines running video with augmented reality to link two objects. Experts can work out obstacles in real-time by connecting physical and virtual reality.

Principles of AR Operation

The AR algorithms allow engineers to combine the sending of running video and pictures in real-time (an image created on the device in the surroundings). The result is a compelling distant access and management solution. Software that is based on augmented reality is more effective than audio communication, video, and graphics transmission. It enables the partner to understand the essence of what is happening and complete the task.   ... " 

Last Cortana Speaker is Recalled

 What seemed like a very credible assistant offering, the Harmon Kardan Intent Speaker, which included the Microsoft assistant Cortana, has left the field.  Further indication that Microsoft is not serious in voice assistants?

The only Cortana-powered speaker is about to lose Cortana

Harmon Kardan will disable the assistant on its Invoke speaker by tomorrow.

By S. Dent     @stevetdent  in Engadget  ... "

Alexa Custom Assistant Announced

 Was expecting something like this, now here it is, will this be widely used?   Implications? Check out what Chrysler is doing.   Introduction below:

Announcing Alexa Custom Assistant

Learn how you can create intelligent assistants tailored to your brand personality and customer needs leveraging Alexa’s advanced AI today. ... 

For the first time, Amazon enables companies to access Alexa’s advanced AI to build their own intelligent assistants with Alexa Custom Assistant; Fiat Chrysler Automobiles is the first Automotive OEM to implement in vehicles

Today, Amazon announced Alexa Custom Assistant, a comprehensive new solution that lets device makers and service providers create intelligent assistants tailored to their brand personality and customer needs. The Alexa Custom Assistant is built directly on Alexa technology, providing companies access to world class, always-improving voice AI technology, customized with unique wake word, voice, skills, and capabilities. The brand’s assistant also seamlessly coexists and cooperates with Alexa, providing customers the benefits of an intelligent assistant that is their product and services expert, while Alexa provides the familiar experiences they already know and love. The Alexa Custom Assistant reduces the cost and complexity of building intelligent assistants into automobiles, consumer electronics, mobile applications, smart properties, video games, and a variety of other digital experiences.   

Global automaker Fiat Chrysler Automobiles (FCA) is the first Alexa Custom Assistant customer, and has already begun the planning process for the development of an FCA-branded intelligent assistant for integration in select vehicle models. The company’s work builds on existing Alexa integrations in its vehicles around the world.... " 

Amazon Opens Assistant AI Tech

 Have now run a car assistant for several years.  After a few bumps it works fine.       Have yet to see any true innovation in car use, just sharing some voice and music services.  What other customizations are useful?   Note the use of the 'Core AI platform',  or 'Alexa Custom Assistant' which implies a deeper technology reach, mentions video games.   Looking for a way to better monetize the AI investment.  See more on the tech at the Amazon Dev Site.       Via Walter Riker

Amazon opens Alexa AI tech for the first time so car makers can build custom assistants

Amazon is letting automakers tailor custom voice experiences using its AI platform  By Nick Statt@nickstatt   in TheVerge

Amazon will now allow third-party companies the unprecedented privilege of accessing the core artificial intelligence underpinning its Alexa digital assistant, a first for the company’s AI platform.

While Amazon has allowed companies to build skills for Alexa and allows pretty much any consumer electronic device maker to integrate Alexa into a compatible product, the e-commerce giant has not licensed the underlying AI tech for use in other assistant-like products. Amazon is calling the new offering Alexa Custom Assistant, and it’s starting out with a focus on the auto market.

Amazon is doing so to allow not just automobile manufacturers, but any company with a need for a digital voice assistant more control over the software experience. This will allow companies to create their own wake words and custom voices and capabilities Amazon says will “co-exist” with Alexa as it’s designed to work today. For the auto market, this provides Amazon the added benefit of having its software built directly into cars. ...'

Also the article mentions that Amazon is making this available for more than just car makers.

Samsung's Home AI Offers to Nag and do General Chores

 CES is virtual this year but is putting out some interesting things, with the usual warning that they are concepts rather than imminent realities.  Looks like Samsung is  touching on the fabled general home-chore butler-helper concept.     Which is hard, a little like General AI  (GAI).    General 'nagging' as proposed though is easy. 

Samsung's latest home robots can do chores and nag you to stop working

Bot Handy's extendable arm can load your dishwasher and set the table.

CES wouldn’t quite feel right without Samsung showing off some robots. A year after it unveiled grapefruit-sized spherical bot Ballie, Samsung has revealed some new machines it’s working on to help out around the home.  ... " 

Alexa Turns Six

Hard to imagine it, Voicebot.ai reports on the birthday and the future of voice assistance.    A big step forward, but what are the other steps we need?

Amazon Echo and Alexa Turn Six, Try to Prepare for Everything

By ERIC HAL SCHWARTZ 

Six years ago, Amazon introduced the world to the Amazon Echo smart speaker and Alexa voice assistant. The AI has become an almost a generic term for a voice assistant and performs tasks from locations far beyond the limits of that first cylinder and pleasant alto voice. In the last year, Amazon launched an entirely new generation of Alexa-enabled devices and expanded into new realms through partnerships with other companies. Alexa meanwhile has added a long list of new and updated features, along with plenty of third-party voice apps. ... "

Continued Rise of Home Assistant Devices

 My array of devices and their use continues to rise, despite the less that 'Smartness' still in evidence.  

Transactions With Voice Assistants on Smart Home Devices Will Hit $164B in 2025: Report

 Eric Hal Schwartz    in Voicebot.ai

Smart home devices will carry out more than $164 billion worth of transactions in 2025, according to a new report from Juniper Research. That’s more than 630% growth in five years from the $22 billion expected this year, the result of more opportunity to use voice assistants to purchase items.

SEEING AND SPEAKING

The enormous spike in transactions carried out with voice assistants can be attributed to a few factors. More people owning smart displays will help boost how comfortable people are with using their voice to make purchases because they can actually see what they are going to buy and observe the process. The report suggests that reaching that number will need connections between smart speakers and smart displays in case the purchase begins as an audio search before the user decides to take a look at the visual and confirm the purchase. Another major driver to voice commerce at home will be smart TVs, for similar reasons. Juniper predicts that more than a fifth of the 2025 value, almost $33 billion, will flow through smart televisions. In comparison, less than 1% of transactions will happen via connected appliances. That’s not surprising as any such appliance like a refrigerator would need to have all of the features of a smart speaker or display to compete with a smart TV.   .... "