Google Duplex Now Available in 49 States

Noted this, have seen Duplex demonstrated, but never used it.  Concern rightly exists about its misuse.  I am sure that Google is thinking through regulatory issues in some detail.

Google’s AI reservation service Duplex is now available in 49 states  in AI News

Google has expanded its controversial AI reservation service Duplex to 49 states across the US.

Duplex will have to comply with privacy regulations which vary between states and – when it expands further outside the US – their national laws too.

Google says the rollout delay in the US was due to awaiting changes in legislation or the need to add features on a per-state basis. Some states, for example, require a call-back number.

The reservation service caused both awe and fear when it was announced in May 2018 for sounding eerily human-like – complete with the “ums” and “ahs” we often fail to avoid – raising concerns it could be used for automating criminal activities such as fraud. ... "

Google Duplex Books Haircuts.

Fascinated by 'booking' as a simplistic form of contextual conversational.   And it is interesting that Google Duplex took so long to get to this variant.  But is that significant?    I can see 'booking' being a script for many kinds of useful interactions.  Certainly as the start of such an interaction.   For example for expert interaction we required the filling out of a problem description form, to determine if an expert was necessary.   Simple example

Google Duplex Can Book Haircuts, 2 Years After Stage Demo  By Eric Hal Schwartz in Voicebot,ai

The Google Duplex voice AI service can now book haircuts for clients, a service Google demonstrated when it announced Duplex in 2018. Duplex uses the Google Assistant AI to call barbers and salons, setting up appointments for their client. Until now, Duplex had been limited to making restaurant reservations or getting store information like opening times on behalf of users.  ... " 

Chatbots Chatting

Now seen this a number of times, never very impressive.

AI News  What happens when Google’s chatty bot chats with a chatbot?

Google Duplex impressed and scared the world in equal parts when it was unveiled, and now we’ve seen how a conversation goes with another chatbot.

Duplex, for a quick primer, is Google’s AI-powered voice bot which can call businesses on a person’s behalf for things such as booking hair appointments. It’s so realistic that everyone has decided that bots must declare themselves as such before chatting with a human.

A company known as PolyAI – which specialises in “enterprise-ready voice assistants” – has posted an account of what happened when Duplex called one of its restaurant assistants.

Duplex was calling businesses over the summer to update opening hours on Google Maps. This is how the conversation went:    ... " 

Turn Your Echo into a Phone

Been placing calls via Alexa for some time, this seems to close the loop.  This seems to open the loop for new kinds of skills that direct, handle, record, manage incoming calls.   A little like Google Duplex?  New abilities for business assistants?  I have an AT&T number so may try this as part of my smart home capabilities.

I saw there are a number of posts here about how Google Duplex links with the Google Assistant ....  so examining that as well.  See the 'Google Duplex Tag' for those posts.

Amazon’s new Alexa partnership lets you link your AT&T number to turn your Echo into a phone

‘Alexa, answer the phone’

By Chaim Gartenberg   @cgartenberg in TheVerge

Amazon is rolling out a new partnership with AT&T today that will let customers link their phone numbers to their Alexa account, allowing them to both make and receive phone calls via their Alexa devices. ... "

Google Assistant and Duplex can Listen Better

This made me think a while about the implications.  It does not necessarily imply a loss of privacy, these are the actions-skills that we accept as extensions to an assistant.   Commands.   So would seem it would be very useful to office voice actions ....   So I could say 'Copy This',  and what follow would be recorded.    Or  'Translate this ... And the following would be translated.     We know that Google can do that ... just need better context operation.    As long as we can control the results.  Lately too have been impressed how the assistant can better handle my mis-stated commands.  But should that make us fear for privacy?

Google Assistant actions can now continuously listen for specific words
Kyle Wiggers@KYLE_L_WIGGERS in Venturebeat

Google today detailed new tools for partners developing on Google Assistant, its voice platform used by over 500 million people monthly in 30 languages across 90 countries. Actions Builder, a web-based integrated development environment (IDE), provides a graphical interface to show conversation flows and support debugging and training data orchestration. Continuous Match Mode allows Google Assistant to respond immediately to a user’s speech by recognizing specified words and phrases. And AMP-compliant content on smart displays like Nest Hub Max speeds up browsing via the web.

Google also revealed that Duplex, its AI chat agent that can arrange appointments over the phone, has been used to update over half a million business listings in Google Search and Google Maps to date. Back in March, CEO Sundar Pichai said Google would use Duplex “where possible” to contact restaurants and businesses so it can accurately reflect hours, pick-up, and delivery information during the pandemic. The company subsequently expanded Duplex in a limited capacity to the U.K., Australia, Canada, and Spain, adding support for the Spanish language in the last instance.  ... " 

Google Using Humans in Duplex too

Reminds me again of our early use of what we called 'concierge' methods for chatbot implementations.  That is, using a combination of logic-AI-learning and humans to provide exactly the right solution, based on context and complexity.    And then evaluating each interaction for later adaptation.  Make lots of sense to tune results to the right resource.  Below also, new hints about what Google is doing with Duplex.

Google Duplex is Using Humans to Make Restaurant Reservations and Not Just AI In Duplex.ai

The New York Times reported yesterday that Google Duplex is using humans to make restaurant reservations, at least in some circumstances. Google Duplex was introduced into four locations in November 2018 and then expanded to 43 states in March 2019. The service can make phone calls on behalf of a user to perform tasks such as making restaurant reservations, setting hair salon appointments, and inquiring about hours of operation for retailers. Making a restaurant reservation is the first of these use cases to roll out broadly and previous reporting confirmed the use of the Google Duplex assistant, an AI-enabled bot, to book tables.

GOOGLE DIDN’T REVEAL THE USE OF HUMANS FOR DUPLEX
However, there was never an indication by Google that humans were in the loop for any of these activities. Google stunned many observers when it demonstrated the service in May 2018 to its I/O developer conference audience. The humanlike sound and speech patterns were better than people expected and the AI appeared to be agile enough to manage even non-linear conversation as long as the domain, such as appointment setting, was limited. Google’s goal is to take care of mundane tasks for users so, in theory, a user would not care whether it is a human or an AI-based bot executing the activity. Using a human is an expensive way to complete the task. The bigger story is that Google was not forthcoming about the state of the technology.   By Bret Kinsella     ... " 

Google Goes CallJoy for Small Business

Google approaches small business to AI automate phone use. like blocking unwanted calls, and providing basic response information about your services and hours.  A concierge like service with a virtual agent to save you time and people resources.   Recording and quality control via search is facilitated.  Nicely thought through.  I can also imagine that they might enhance this with more general chatbots.    Can this be abused, as was the worry with the Duplex system they demonstrated?

Helping small business phones get smart with CallJoy
By Bob Summers,   Google Blog
General Manager, CallJoy
Published May 1, 2019

Think about how many times you’ve called a small business lately. I call local businesses near my home and office all the time—just last weekend, I was on the phone with the nearest exotic pet store to see if they had food in stock for our family's pet lizard.

My team within Area 120, Google’s workshop for experimental projects, conducted testing and found that small businesses receive an average of 13 phone calls every day. If you apply that average to America's 30.2 million small businesses, that would equal roughly 400 million incoming daily calls to local businesses from consumers placing a to-go order, booking an appointment, inquiring about inventory and more. That’s why we built CallJoy, a cloud-based phone agent that enables small business owners to measure, improve and automate customer service.

Meet CallJoy

With CallJoy, small businesses have access to the same customer service options that have historically only been available to larger corporations. If you’re associated with small business using CallJoy, here’s how it works: After a quick setup, you’ll receive a local phone number. CallJoy will immediately begin blocking unwanted spam calls so you receive the calls that matter—the ones from customers. Then, when the phone rings, the automated CallJoy agent answers, greets callers with a custom message and provides basic business information (like hours of operation).

If the customer calling would like to complete a task which can be done online, like place a to-go order or book an appointment, CallJoy’s virtual agent will send the customer an SMS text message containing a URL. Whether the customer speaks with you, talks to an employee, or just interacts directly with the CallJoy agent, the call will be recorded and transcribed for quality purposes. This allows small business owners to tag and search each conversation based on topic. For example, a hair salon owner can search how many times a day callers ask about “men’s haircut pricing” or “wedding hairstyles.” From here, CallJoy compiles your data in an online dashboard and emails you a daily update, which includes metrics like call volume and new versus returning callers.  .... " 

Google Duplex Calls are often From Humans

Much akin to our own attempts at putting people and/or humans in a conversational path, and looking for signals in the interaction to determine when you should shift from one kind of agent to the other.

I assume almost always bailing to a human agent.  Very natural idea.  So whats the beef?  Are you fooling 'some of the people some of the time', and detecting if they get alarmed or confused or angry if they detect they are fooled?

I would mainly be alarmed that there would be many more calls, like we have been seeing lately, because robocalls are cheaper to deliver than using people.    A 'concierge model' like we used used a clearly robot agent, which then passed people off to people or a chatbot. 

In The Verge:
A quarter of Google Duplex calls are actually placed by humans
If the AI sounds eerily human, well, that’s because they just might be    By Natt Garun@nattgarun 

Earlier this month, I shadowed several restaurants throughout New York and talked to restaurant employees across the US to see how they’ve received Google Duplex, the AI that makes life-like calls for reservations on your behalf. Most agreed that the AI sounded unmistakably human — and according to Google’s response to reporting by The New York Times, there’s a 25 percent chance that they were.

Google says that a quarter of Duplex calls start with human callers, and 15 percent start with the AI and are later intervened by a person from the Duplex call center. The company told The New York Times that it uses a variety of signals to decide whether a call should be placed by a human or a robot, “like if the company is unsure of whether the business takes reservations, or if the user of the assistant might be a spammer.”   ... ' 

Considering Duplex. Block or Receive with a Bot?

Good piece in the Verge about Google Duplex.   And how businesses can opt out of incoming calls from it.  Duplex lets you set up a kind of  agent with a script to do tasks.   An example to start with could be making a reservation at a restaurant.   Another might be asking if a particular product is in stock at a location.   Or asking for information about the status of an order.   All with voice interaction. As I understand it the calling agent will always identify itself as an AI.  The fear is that this may get out of hand and these will turn into artificially  generated bot  'robocalls' and tie up the time of small business.  Ultimately too, the receiving call could also be handled by an agent.    In a sense either side of call could be either a bot agent or a person.  Good detail in the article linked to:

How to opt out of automated Google Assistant calls to your business
Reject the Duplex vex         By Natt Garun  @nattgarun in TheVerge

Somewhat buried in a piece I recently wrote about how restaurants have been experiencing Google Duplex — the voice-calling AI that makes reservations on a customer’s behalf — is how the company automatically opts businesses into receiving these types of robocalls if they have a Google business page. While this might be fun for you if you’re interested in hearing what Google Duplex’s strikingly human AI sounds like, you may not want to participate in Google’s machine learning initiative without expressed consent.

Google says its AI may call businesses to make and confirm reservations or inquire about business hours. If, for whatever reason, you want to opt out of receiving calls from Google Assistant, here’s how to ensure you’re blocking these particular robocalls.  .... "

Google Duplex Makes Real Reservations

Google Duplex gets closer to what we would call a human assistant.   You give it a task by voice, with Google assistant, and it does the work.    Here the work is making calls to restaurants to ensure  reservations.  The  AI asks and answers clarifying questions of both the requester and the restaurant.  The goal is to get a confirmation.

 The 'creepiness' that is perhaps included is that the restaurant does not know its an AI voice, though in the test they are participating.   Later Google implied the system would make that clear when calling.   The video on the link below shows how it would work.

This kind of chatbot interaction obeys a 'script'.  We have all internalized such scripts for dealing with people, or machines.   If either party asks questions that are not interpretable in the script, we get confused too.   And perhaps pass the person off ...  to another script, or to some other resource in concierge fashion, which is what we did in our own brand experiments.

I like that they are doing this, if its useful people will get used to this idea of talking to a machine in new and more flexible ways.   We have learned that already in just the past couple of years.  Even when a machine starts the conversation.

Google brings its Duplex AI restaurant booking assistant to 43 states By Brian Heater@bheater in TechCrunch

No moment wowed the audience at last year’s I/O more than Duplex. The demo of the artificial intelligence restaurant and appointment booking program left many in the audience wondering whether Google had just pulled a fast one over on them.

Turns out, it’s real.

Over the summer, I got a chance to test drive Duplex at a Thai restaurant in Manhattan. And later in the year, the company rolled the program out in limited testing to restaurants in four U.S. cities. Today, it announced that it’s opening things up even more.  ... " 

Future of Voice with Human Mimicking

Was recently asked about voice based systems that mimic/fool people into thinking the speaker is human.  Notably for both assistant and possible marketing applications.  Notably the 'Google Duplex' method demonstrated this year.  This article in theVerge outlines their possible issues and next steps:

Google is being vague with disclosure in early real-world Duplex calls    By Chris Welch@chriswelch  

Google has begun rolling out its futuristic Duplex feature, which can automatically make voice calls to restaurants and other businesses on a user’s behalf, to a small group of Pixel owners in “select” cities around the US. VentureBeat managed to test out Duplex in the real world and recorded what the experience is like when initiating a call through Google Assistant. That part seems fairly straightforward. But the exchange between Duplex and a restaurant on the other side of the call is raising some early concerns about transparency.   .... " 

Alibaba has a Better Intelligent Assistant than Google's

In particular the claim that it is more conversationally powerful is interesting.  Such advances could lead to deeper interaction with humans and further engagement.

Alibaba already has a voice assistant way better than Google’s
It navigates interruptions and other tricky features of human conversation to field millions of requests a day   by Karen Hao  in Technology Review excerpt: 

In May, Google made quite the splash when it unveiled Duplex, its eerily humanlike voice assistant capable of making restaurant reservations and salon appointments. It seemed to mark a new milestone in speech generation and natural-language understanding, and it pulled back the curtain on what the future of human-AI interaction might look like.  Still only in Chinese. 

But while Google slowly rolls out the feature in a limited public launch, Alibaba’s own voice assistant has already been clocking overtime. On December 2 at the 2018 Neural Information Processing Systems conference, one of the largest annual gatherings for AI research, Alibaba demoed the AI customer service agent for its logistics company Cainiao. Jin Rong, the dean of Alibaba’s Machine Intelligence and Technology Lab, said the agent was already servicing millions of customer requests a day.

The demo call involved the agent asking a customer where he wanted his package delivered. In the back-and-forth exchange, the agent successfully navigated several conversational elements that demonstrated the breadth of its natural-language capabilities.

Take this exchange at the beginning of the call, translated from Mandarin: ... "

Proof of Concept Malware Uses Intelligence

I know executives who are alarmed when they see programmers or major companies outline how new kinds of Malware can be built.  Isn't it bad enough?  But it does help to outline the vulnerabilities to plan for them. This example might be Outlined:  If we can use intelligence to feel you out and trick you as to who we are , we can more likely be successful.    All Phishing does this.   Recent demos of Google Duplex hint at this.   Somewhat unusual to see IBM doing this.   Expect to see more of this.

IBM's proof-of-concept malware uses AI for spear phishing   in V3
The neural network running DeepLocker hides its intent until it finds the right victim  The world is beginning to transition from the cloud era to the artificial intelligence (AI) era, as systems and networks grow and learn. But just as the web and cloud eras had their own threats, the same applies to this new landscape - and it is AI itself.

Excitement and confusion abound over AI, but despite - or perhaps because of - this, the technology can pose a real danger to computer users.

"As machine learning matures into AI, nascent use of AI for cyber threat defense will likely be countered by threat actors using AI for offense," Rick Hemsley, managing director at Accenture Security, told us earlier this year.

So on the face of it, IBM's development of DeepLocker - ‘a new breed of highly targeted and evasive attack tools powered by AI' - seems like it sets a dangerous precedent.

There is method to the madness. IBM reasons that cybercriminals are already working to weaponise AI, and the best way to counter such a threat is to watch how it works ... " 

Here is the report on Deeplocker by IBM.

Duplex AI and the Call Center

Google is for now denying that Duplex will take over the enterprise call center.  And you should ask if just sounding and acting like a human is enough to make such a chatbot more valuable.   As I have often said, having a memory, an architecture for solving problems in context,  a means to adapt to the interaction and continuous updating from history,  are probably most useful.  And of course, also handing off a problem to any agent, human or otherwise, who is more likely to solve the problem.    Fooling the caller into thinking the chat agent is human is not by itself very useful.  Google, please do keep working on it, and place it on your assistants to test.

Google’s Duplex AI could kill the call center in Quartz by Dave Gershgorn

The robots on the other side of the customer support line could soon start to sound a lot more human.

Google is reportedly shopping its Duplex AI system around as a tool for call centers, according to The Information, including a large insurance company.

Duplex would handle simple calls for the insurance company, and if the customer started asking complex questions the bot can’t handle a human would step in, according to the report. However, it’s unlikely that AI research will cease after mastering simple conversations, meaning call centers could one day be largely automated using this technology. ... " 

Digital Trends looks at Google Duplex

A yet closer look at the controversial approach.  My opinion, this attempts to make the conversation and its components and goal directions autonomous.   That is impressive.  This makes it more impressive, but also more likely to fool a human, if they don't notice any warning given.   Tne unintended consequences of that remain unclear.    Ordinary chatbots, such as those we created,  were not that flexible. 

What is Google Duplex?  By  Erika Rawes  @Erika_Tin  in Digital Trends

During Google I/O 2018, Google revealed “Google Duplex” to the world. The technology, which is both impressive and a bit on the creepy side, featured a human-sounding robot having a conversation with a person who couldn’t even tell that they were talking to a robot. The demonstration freaked some out but impressed others, including our Mobile Editor Julian Chokkattu, who got a chance to demo Duplex recently.

Google Duplex is a pretty big leap the evolution of artificial intelligence. Now, this is not to say that we’ll have human-like robots that can do the laundry or go shopping like in that movie I, Robot (at least not anytime soon). But, Google Duplex is a huge step in terms of A.I.’s ability to more naturally converse with humans. But what is it?  .... "

And Language Pros on Google Duplex

Inevitable that we will be doing lots more talking to machines in the future, get ready for it.  Good read:

What natural language pros are saying about Google’s Duplex

We may now be having the first of many public conversations about how AIs should talk to humans.
By Mark Sullivan in Fast Company

Google’s much-hyped reservation-setting AI, Duplex, is a one-trick pony with nowhere near the conversational versatility we’ll see in the artificial general intelligence (AGI) of the future. But the natural language interface it employs is provocative because it suggests the way we might interact with the AGIs of the future. .... " 

Google Re-demos its Duplex Chat, with Warnings.

Clever look at a chatbot that has a goal of delivering a phone goal.    But when it was first shown it was clear it could fool and ultimately very much annoy the human in the loop.   Is it do no evil, or fool no human?  So now it been repositioned.   Still, though, will we accept being in this no loop?

Google gives its Human-like Phone Chatbot a Demo Redo  by Lauren Goode in Wired

WHEN GOOGLE FIRST demonstrated its AI phone-calling technology Duplex back in May, the pre-recorded demo struck many observers as eerie. Piped through the speakers on stage at the Google I/O developer conference while a video capture of an Android phone played on screen, we heard an artificial voice call both a hair salon and a restaurant to book reservations on behalf of a human.

Right away, many in the tech community cited two big problems. First, the people on the receiving end of the call were unaware that the voice speaking into the phone was a machine, meaning Duplex was essentially fooling unsuspecting humans. Second, the bot in the demo never indicated it was recording the phone call, raising the eyebrows of privacy advocates and prompting follow-up questions from journalists (including writers at WIRED).

On Tuesday, Google gave multiple demonstrations of its Duplex technology in action. This time, there were some obvious differences.

Now, just a couple weeks ahead of Duplex’s rollout among a small set of users and businesses, Google is trying to give its phone-calling robot a do-over. The company is attempting to prove it has addressed some of the concerns about Duplex. And its latest pitch around transparency is coming at a time when some of its more critical use cases for AI are being seriously questioned—just recently, the company released a set of AI principles prohibiting Googlers from using AI in technologies that could violate human rights or cause “overall harm.”

On Tuesday, at a hummus shop in Mountain View, California just down the road from Google’s headquarters, the company gave multiple demonstrations of its Duplex technology in action. This time, there were some obvious differences. “Hi, I’m calling to make a reservation,” the bot said, which Google patched through speakers in the shop so the assembled reporters could hear it. “I’m Google’s automated booking service, so I’ll record the call. Can I book a table for Thursday?”  ... " 

Microsoft Makes Chat calls in China

More word of  call-making chatbots, akin to recently announced Google Duplex.  Ultimately you will have to have such systems communicating, with people and other systems, but the implications need to be thought through.  Will we always know who is calling?

Microsoft also has an AI bot that makes phone calls to humans
Similar to Google Duplex, but only in China
By Tom Warren  ... in theVerge

 " ... Google demonstrated a jaw-dropping new capability in Google Assistant earlier this month, allowing the Assistant to make calls on your behalf. While Google Duplex generated controversy and discussion around artificial intelligence, Microsoft has been testing similar technology with millions of people in China. At an AI event in London today, Microsoft CEO Satya Nadella showed off the company’s Xiaoice (pronounced “SHAO-ICE”) social chat bot.

Microsoft has been testing Xiaoice in China, and Nadella revealed the bot has 500 million “friends” and more than 16 channels for Chinese users to interact with it through WeChat and other popular messaging services. Microsoft has turned Xiaoice, which is Chinese for “little Bing,” into a friendly bot that has convinced some of its users that the bot is a friend or a human being. “Xiaoice has her own TV show, it writes poetry, and it does many interesting things,” reveals Nadella. “It’s a bit of a celebrity.” ... " 

More on Google Duplex

In the Google AI blog.   A more lengthy and technical view of getting tasks done by phone conversations.

Google Duplex: An AI System for Accomplishing Real-World Tasks Over the Phone

Posted by Yaniv Leviathan, Principal Engineer and Yossi Matias, Vice President, Engineering, Google 

A long-standing goal of human-computer interaction has been to enable people to have a natural conversation with computers, as they would with each other. In recent years, we have witnessed a revolution in the ability of computers to understand and to generate natural speech, especially with the application of deep neural networks (e.g., Google voice search, WaveNet). Still, even with today’s state of the art systems, it is often frustrating having to talk to stilted computerized voices that don't understand natural language. In particular, automated phone systems are still struggling to recognize simple words and commands. They don’t engage in a conversation flow and force the caller to adjust to the system instead of the system adjusting to the caller.  .... "

Google Duplex Calling Will Let You know its an AI

The presentation of the Google Duplex calling service, demonstrated this week and planned for this summer, impressed many as a great idea.   A system that could construct a goal-driven interaction with other people or devices.   So it could, for example, make a reservation at a restaurant, or make complex travel reservations.   Even negotiate something online.   Along the way it was hinted it might solve the Turing AI test, and appear to be human, with a pleasing voice and some contextual reasoning power.  But others pointed out that this could lead to a kind of super robocalling capability,  leading to potentially nightmarish results.    Caution ...

Google: Duplex phone calling AI will identify itself  ... You'll know if you're talking to AI -- we think.    By Richard Lawler, @Rjcc in Engadget: 

Google is then quoted as clarifying this:

" ... We understand and value the discussion around Google Duplex -- as we've said from the beginning, transparency in the technology is important. We are designing this feature with disclosure built-in, and we'll make sure the system is appropriately identified. What we showed at I/O was an early technology demo, and we look forward to incorporating feedback as we develop this into a product. ... "