AI Tools for Skill Improvement

 Chess and other game playing as a means to have access to tools to improve skills.  How does this translate to more general skills?

AI Helps Humans Level Up Chess players and programmers now have access to tools that can improve skills and unleash creativity   By Harry Goldstein  In IEEEiOrg

BACK IN THE mid-1970s, IEEESpectrum senior editor Phil Ross played one of the first chess programs capable of vanquishing humans. He capitulated quickly—too quickly, it turned out: Although the program that beat him was good at openings and the middle game, it was terrible at the end game. Fast forward 50 years and the highest-ranked chess players in the world are AIs, with humans trailing far behind.

As Ross points out in his online piece “ AI’s Grandmaster Status Overshadows Chess Scandal,” the recent brouhaha involving world champion Magnus Carlsen and up-and-comer Hans Niemann highlights how chess-playing AIs, referred to as “engines” by the cognoscenti, have overtaken humans in terms of raw game-playing accuracy. The scandal also shows how AIs are being used by players at all levels to get better faster, fostering a boom in the sport.

If you were born a few decades ago, your best shot at playing and learning from grandmasters was either to be lucky enough to know one or to somehow qualify for the high-level tournaments in which they participated. Nowadays, chess newbies can log in and play engines that far exceed their own abilities, learning strategies and moves in days or weeks that in the past might have taken months or years. Engines can also help neophytes and grandmasters alike analyze their own games to give them an edge against human opponents. 

In fact, if you burrow down the rabbit holes of chess YouTube or Twitch, you’ll find grandmasters giving move-by-move analyses of games that engines played against each other. These AI tools, along with the humans who use them, are readily accessible: You can play the cybernetic versions of super grandmasters, like the open-source, current top-ranked chess engine Stockfish 14.1, not to mention tens of millions of human opponents on sites like Chess.com, a global community of some 93 million players and a central player in the cheating controversy.

While it is certainly true that bad actors use AI to cheat at chess—potentially posing an existential threat to the game, as Carlsen has suggested—it is equally true that the chess world has openly embraced AI and has been thriving as a result. Similar risk/reward calculations will need to be made in other domains.  .... ' 

Making Pizza etc. with Robots

Here a long time amateur cook, very interested in how this could all be well automated.  Starting to see it in fast food. Like also the connection to things like clothing.  

Solving the challenges of robotic pizza-making

A new technique could enable a robot to manipulate squishy objects like pizza dough or soft materials like clothing.

Adam Zewe | MIT News Office   Publication Date: March 31, 2022

Imagine a pizza maker working with a ball of dough. She might use a spatula to lift the dough onto a cutting board then use a rolling pin to flatten it into a circle. Easy, right? Not if this pizza maker is a robot.

For a robot, working with a deformable object like dough is tricky because the shape of dough can change in many ways, which are difficult to represent with an equation. Plus, creating a new shape out of that dough requires multiple steps and the use of different tools. It is especially difficult for a robot to learn a manipulation task with a long sequence of steps — where there are many possible choices — since learning often occurs through trial and error.

Researchers at MIT, Carnegie Mellon University, and the University of California at San Diego, have come up with a better way. They created a framework for a robotic manipulation system that uses a two-stage learning process, which could enable a robot to perform complex dough-manipulation tasks over a long timeframe. A “teacher” algorithm solves each step the robot must take to complete the task. Then, it trains a “student” machine-learning model that learns abstract ideas about when and how to execute each skill it needs during the task, like using a rolling pin. With this knowledge, the system reasons about how to execute the skills to complete the entire task.

The researchers show that this method, which they call DiffSkill, can perform complex manipulation tasks in simulations, like cutting and spreading dough, or gathering pieces of dough from around a cutting board, while outperforming other machine-learning methods. .... '   

Amazon Alexa Contextual Skills Kit

Amazon skill development reports a new set of area specific skills for use.  How well does this support contextually good assistance?   Taking a closer look.  More at the link below.

Build a Voice Skill That’s Just Right for You  .... 

With the Alexa Skills Kit, you’re only limited by your imagination. You can build engaging, interactive voice experiences for everything from games and food to Smart Home devices. Our built-in voice interaction models make it easy.

Custom Skills: Use your imagination to create your own voice experiences

Game Skills: Build everything from interactive adventures to quizzes

Music & Audio Skills: Empower listeners to access your audio streaming services

Food Ordering Skills: Enable hungry customers to find restaurants and order

Smart Home Skills: Enable customers to control your cloud-connected devices

Video Skills: Let customers control your video device and content

Flash Briefing & News Skills: Make it easy for customers to get your latest content updates

Connected Vehicle Skills: Enable customers to control their connected vehicles

Knowledge Skills: Build Q&A skills in minutes without writing code

Get started »

Trade Skill Schools and Pandemic

Virtual reality is not near enough as yet.   So how do we teach physical skills?   I was asked in recent days by a training manager of electricians.  Quickly,  because we need them now. 

Virtual class was a devastating blow to trade school students  in Fastcompany

For students learning trades like carpentry, cosmetology, or automotive technology, in-person class time is crucial. School closures hit them hard.

Mark Chaney hates that the pandemic has forced the Buckeye Hills Career Center in Rio Grande, Ohio, where he teaches to still have a schedule with students in school only part time. That may work for English and math classes during the pandemic, he said, but his students are trying to learn physical skills, not just intellectual ones. They need to handle, build, and take apart pipes, ductwork, and breaker boxes every day, not spend half their week doing online work at home.  ... " 

Alexa Skills Update

 From the Amazon Ops site, gives some indication of what they are heading towards regards skills development.  A many page list of examples follows at the link.   With extensive writeups of the skills

Alexa Skills Kit Feature Updates

On this page: Feature Updates    |    Locale Availability

Feature updates are categorized into the following types of releases:

Preview: This feature is not immediately available for all developers. Developers are required to sign up and agree to the terms of the release to use.

Beta: This feature may lack important functionality and may have future breaking changes, but is operational and available to all developers.

Generally Available: This feature is immediately available for all developers in all commercial regions (unless otherwise noted), and developers can use it to run production workloads. : ...'

Learning Better with Shape-Shifting

 Like the example of modifying objects to address specific elements of training.   Unusual application, uses outside of sports?

Better Learning with Shape-Shifting Objects

MIT News, Adam Conner-Simons

Shape-shifting objects that can help users improve their skills is an area of investigation for Massachusetts Institute of Technology (MIT) researchers, who conceived of a basketball hoop that trains players more effectively by shrinking and raising to help them make shots more consistently. Experiments demonstrated that training on the auto-adaptive hoop improved player performance more than using a static hoop or the manually-adaptive mode. Autodesk's Fraser Anderson said, "You don't have to rely on your own sense of whether or not you've mastered a skill: the system can do that and take out the self-doubt, overconfidence, or guesswork."   ... '

Alexa Guesses Intent, Latent Goals

 Quite interesting,   but this depends on the results.  Admirable idea.   Based on what I read it seems this will at least somewhat depend on suggesting existing skills that address goals and needs.    And I still don't see how these skills will fill in the gaps that might exist.    Will be watching.

Alexa gets smarter as Amazon boosts AI models with ‘latent goal’ detection   By Maria Deutscher in SiliconAngle

Amazon.com Inc. is enhancing Alexa with new artificial intelligence models that can help users find useful Skills, or apps, to install on their smart speakers.

Alexa Skills are created by third-party developers and provide features not included by default in the voice assistant, such as the ability to control smart lights or tune into a local radio station. There are more than 100,000 free and paid Skills available.

Amazon says Alexa can can now suggest Skills that it believes a user may find handy by inferring “latent goals” from their commands. For example, if a user asks Alexa a history question, the voice assistant might reply with the desired information and then follow-up by asking whether they’d like to download a Skill that can launch history quizzes. When Alexa is asked to find the stock price of a company, it could recommend a financial Skill that can provide additional information on the markets. ,, "

The Need to become an Adaptive Expert

Been known for a long time, soft skills are important.  Expertise is by definition a means of changing to meet the context. 

Becoming an 'Adaptive' Expert
By Quinn Burke, Cinamon Sunrise Bailey
Communications of the ACM, September 2020, Vol. 63 No. 9, Pages 56-64
10.1145/3366171

In today's software development industry, jobs have become more cognitively complex and require workers who are more collaborative and creative in their problem-solving techniques.14 Employees also must be able to combine diverse specializations rather than just having routine knowledge in one domain.22 While the "hard" technical skills associated with programming remain a prerequisite for new hires, the industry also wants software developers who can readily demonstrate a range of so-called "soft" skills, including the capacity to communicate clearly, facilitate an open and inclusive workplace environment, and demonstrate the resiliency and flexibility to work on a range of tasks.24

Our own past research4 interviewing software industry hiring managers indicates that discerning such soft skills among new hires is an overwhelming priority across companies. The industry hiring managers and directors we interviewed over the past two years stated that while the capacity to code is a necessity for employment, these managers actually spend the vast majority of their recruitment time assessing a candidate's soft skills, as these suggest the presence of adaptive expertise (AE) and the candidate's potential for persistence and continual learning on the job.4 What was also intriguing to us in discussion with a wide range of hiring managers was their expressed willingness to consider graduates from alternative educational settings—in particular, so-called "coding bootcamps"—alongside more traditional hires from undergraduate computer science (CS) programs.4 While there is no single representative model of a coding bootcamp, these intense training programs extend, on average,14 weeks in duration, cost approximately $12,000, and emphasize teaching the programming skills that employers look for from new software developer hires (particularly front-end programming) while also enabling their graduates to grasp the most essential aspects of coding.6 Much of this expressed willingness to hire codecamp graduates stemmed directly back to hiring managers' perceptions that what boot-camp students may lack in rigorous CS knowledge is counterbalanced with greater work experience and the interpersonal and intrapersonal skills to join a wider team while remaining resilient in the face of unexpected challenges.

Microsoft Backing from Cortana

Microsoft seems to continue to fall back from having Cortana  as a general assistant.  but has implied they want to tp continue it as a  productivity aid for their business tools.   They are removing it from a number of well known third parties.   Decreasing support for skill development.   Would think they would continue to keep the presence to learn from it.

Microsoft will shut down the Cortana iOS and Android apps in 2021
It will also remove the digital assistant from Harman Kardon Invoke speakers.
Igor Bonifacic, @igorbonifacic    in Engadget

We already knew Cortana’s days as a consumer-facing digital assistant were numbered after Microsoft said earlier this year it would remove the AI from its Android launcher app. But the company has now detailed additional cuts that users are likely to feel more keenly.

The changes won't happen all at once. To start, Microsoft will end support for third-party Cortana skills on September 7th, 2020. In early 2021, Microsoft then plans to discontinue the Cortana apps on iOS and Android, as well as remove the current Cortana functionality the first-generation Surface Headphones feature. Sometime in early 2021, Harman Kardon Invoke speakers will lose access to the digital assistant as well.   ... " 

Reinforcement Learning for Skill Discovery

Can skills be dsicovered.  That is, a means to find better behavior that leads to prescribed real-world goals?   Here in the Google Research blog,  addressing the use of unsupervised reinforcement learning (RL). Note the determination and inclusion of constraints.   Like in classic optimization problems. Largely technical, but thoughtful positioning.  Considerable links in the article below.

DADS: Unsupervised Reinforcement Learning for Skill Discovery
Friday, May 29, 2020
Posted by Archit Sharma, AI Resident, Google Research

Recent research has demonstrated that supervised reinforcement learning (RL) is capable of going beyond simulation scenarios to synthesize complex behaviors in the real world, such as grasping arbitrary objects or learning agile locomotion. However, the limitations of teaching an agent to perform complex behaviors using well-designed task-specific reward functions are also becoming apparent. Designing reward functions can require significant engineering effort, which becomes untenable for a large number of tasks. For many practical scenarios, designing a reward function can be complicated, for example, requiring additional instrumentation for the environment (e.g., sensors to detect the orientation of doors) or manual-labelling of “goal” states. Considering that the ability to generate complex behaviors is limited by this form of reward-engineering, unsupervised learning presents itself as an interesting direction for RL.

In supervised RL, the extrinsic reward function from the environment guides the agent towards the desired behaviors, reinforcing the actions which bring the desired changes in the environment. With unsupervised RL, the agent uses an intrinsic reward function (such as curiosity to try different things in the environment) to generate its own training signals to acquire a broad set of task-agnostic behaviors. The intrinsic reward functions can bypass the problems of the engineering extrinsic reward functions, while being generic and broadly applicable to several agents and problems without any additional design. While much research has recently focused on different approaches to unsupervised reinforcement learning, it is still a severely under-constrained problem — without the guidance of rewards from the environment, it can be hard to learn behaviors which will be useful. Are there meaningful properties of the agent-environment interaction that can help discover better behaviors (“skills”) for the agents?  ... "   ... ' 

AI as Workplace Disrupter

Expected for jobs that require high skills, but these same jobs will morph to related tasks as well.  Being better skilled usually means you have the ability to re-train.   And today there are many more ways to re-train and become certified in new areas.

AI is the Next Workplace Disrupter, and It's Coming for High-Skilled Jobs
The Wall Street Journal
Eric Morath
February 23, 2020

A Brookings Institution study predicts that the next wave of artificial intelligence (AI)-driven automation will target white-collar professions like marketing specialists, financial advisers, and computer programmers. AI enables computers to analyze data, anticipate outcomes, and learn from experience by recognizing patterns and making decisions—jobs typically performed by highly skilled, well-educated employees. Stanford University's Michael Webb used AI to review more than 16,000 AI-related patents to ascertain capabilities like disease diagnosis or object recognition, while examining a U.S. Labor Department database of occupations to catalog specific tasks needed for jobs. By matching the frequency of overlap between these two factors, Webb found that holders of bachelor's degrees are five times more likely to be impacted by AI than those with high-school diplomas. AI may potentially automate certain tasks for some professionals without entirely replacing them, while others could find their jobs simplified and easily filled by less-educated workers. ... '

Stats on Alexa Skills

Decrease in skill increase.  Lots of other interesting skills, in particular by language. Now lets get the quality and intelligence up

New Alexa Skill Data Show New U.S. Skills Launched in 2019 Fall to Lowest Level Since 2016
Bret Kinsella in Voicebot.ai

U.S. Alexa skills climb to 70,729, up 24.6% for the year, but the smallest total increase in skills since 2016, with only 38.2 new skills added per day in 2019 compared to 85.0 per day in 2018

There were fewer Alexa skills generated in the U.S., UK, and Germany in 2019 than in 2018

Reduction in developer incentives appears to be a contributing factor in reducing new Alexa skill introductions

New Alexa skill growth is strong in Spain and Italy with the former showing 200% growth in the second half of 2019 alone

Amazon announced in September that there are more than 100,000 unique Alexa skills worldwide. The largest of those markets, the U.S., had 70,729 alone at year-end followed by other English-speaking countries the UK at 36,341, India at 32,879, Canada with 25,950, and Australia 24,651. 

The combined numbers will tally far beyond 100,000 because these figures represent the number of Alexa skills available in January 2020 to users in those countries and some skills, particularly English-language skills, are published simultaneously in multiple locations. The largest non-English Alexa skill store today is German with 10,065 skills. .... " 

Defining the Smart Home: More than a Speaker

Very true, but it sets the stage.  There is so much more work to be done to establish the level  assistance that needs to created.   Skills are many, but most are trivial.    Below article points to recent surveys. 

ProBeat: A smart speaker does not a smart home make in Venturebeat, from CES By  Emil Protalinski

To coincide with CES 2020 this week, Strategy Analytics released a survey claiming that smart homes had passed “the tipping point” and that “Most Homes Are Now Smart Homes.” Putting aside that U.S. homes don’t represent the world’s homes, the survey assumed that you had a smart home if you “own at least one smart home device.” First, owning doesn’t mean actively using. Second, a smart home is not defined by owning “smart speakers, interactive security systems, and smart thermostats.” Third, that begs the question: How should we define a smart home?

The smart home definition is important for a few reasons. Technology has been flooding into our homes, and CES 2020 showed this trend is not abating. Internet of things (IoT) devices are not going away, and neither is the dream of a smart home. That doesn’t mean all home devices marketed as “smart” are inherently useful, however, let alone make our homes “smart.” Additionally, there are plenty of valid privacy and security concerns in a potential smart home....." 

A Look at the Smart Speaker Market

Been asked to look at how 'smart speakers' and Assistant market is evolving.  And how changes in tech  will influence. Useful stats below.  Still lots of entry-level sales. Notable other results:  Relatively little buying is being done with them.  Though some 80K skills have been written for Alexa, but  no major successes. (The Bloomberg article linked to here is a good look at this, where are the Apps that emerged with smartphones?)

Alexa devices maintain 70% market share in U.S. according to survey
More than 50% of sales are for entry-level devices: Echo Dot and Google Home Mini.   By Greg Sterling on August 9, 2019 at 9:47 am

In January, 2018 Consumer Intelligence Research Partners (CIRP) estimated that Amazon Echo/Alexa controlled 69% of the U.S. smart speaker market. Google Home owned the remainder, with a tiny percentage going to Apple’s HomePod. More than a year later, new CIRP data shows roughly the same breakdown, with Amazon maintaining a 70% market share.

76 million units. Based on its newest survey, the firm said “the US installed base of smart speaker devices is 76 million units, up from 70 million units in the March 2019 quarter and 50 million units in the June 2018 quarter.” Other surveys argue there are more than 100 million units in American homes.

U.S. smart speaker market share (millions)  ... " 

Improvements for Future Voice Assistants

Survey of future improvements that could be useful.  Not complete or specific to voice assistants.   But useful thoughts.

5 ways that future A.I. assistants will take voice tech to the next level   By Luke Dormehl in DigitalTrends

“Before Siri, when I talked about [what I do] there were blank stares,” Tom Hebner, head of innovation at Nuance Communications, which develops cutting edge A.I. voice technology, told Digital Trends. “People would say, ‘Do you build those horrible phone systems? I hate you.’ That was one group of people’s only interaction with voice technology.”

That’s no longer the case today. According to eMarketer forecasts, almost 100 million smartphone users will be using voice assistants by 2020. But while A.I. assistants are no longer a novelty, we’re still at the start of their evolution. There’s a long way to go before they fully live up to the promise that voice assistants have as a product category.

Here are five ways in which the technology could improve to make it smarter and more efficient — and help us lead more productive lives as a result. Call them “predictions” or a “wishlist,” these are the challenges that need to be solved.  .... " 

Alexa Determining Skills for Customer Needs

Its ultimately a concierge type problem.   How do we determine the best augmenting skill?    Specified need, context, technical ability,  past interactions .... Lots can be at play.  Ultimately a component of all intelligent conversations with a goal.

Rank: How Alexa Determines What Skill Can Best Meet a Customer’s Need    By Young-Bum Kim  Amazon Developer

Amazon Alexa currently has more than 40,000 third-party skills, which customers use to get information, perform tasks, play games, and more. To make it easier for customers to find and engage with skills, we are moving toward skill invocation that doesn’t require mentioning a skill by name (as highlighted in a recent post).

To enable name-free skill interaction, Alexa currently uses a two-step, scalable, and efficient neural shortlisting-reranking approach. (I described our approach to shortlisting in a post yesterday). The shortlisting step uses a scalable neural model to efficiently find the optimal (k-best) candidate skills for handling a particular utterance; the re-ranking step uses rich contextual signals to find the most relevant of those skills. We use the term “re-ranking” since we improve upon the initial confidence score provided by the shortlisting step.

This week, at the Human Language Technologies conference of the North American chapter of the Association for Computational Linguistics (NAACL 2018), my colleagues and I presented a paper, “A Scalable Neural Shortlisting-Reranking Approach for Large-Scale Domain Classification in Natural Language Understanding,” that describes our approach.

hyprank_overview.png
A high-level flow of the two-step shortlisting-reranking approach

The Challenge
The problem here is essentially a domain classification problem over the k-best candidate skills returned by the shortlisting system, which we call Shortlister. The goal of Shortlister is to achieve high recall — to identify as many pertinent skills as possible — with maximum efficiency. On the other hand, the goal of the reranking network, HypRank, is to use rich contextual signals to achieve high precision — to select the most pertinent skills. Designing HypRank comes with its own challenges:

•    Hypothesis representation: It needs to use available contextual signals to produce an effective hypothesis representation for each skill in the k-best list;
•    Cross-hypothesis feature representation: It needs to efficiently and automatically compare features, such as a skill’s intent confidence, to those of other candidate skills in the k-best list; 
•    Generalization: It needs to be language-agnostic; and
•    Robustness: It needs to be able to accommodate changes, such as independent modifications to Shortlister or to the natural-language-understanding models that provide skill-specific semantic interpretation of utterances. .... "

AI to Drive Re-Skilling Efforts

Makes sense,  but the primary challenge will be how these skills are interlinked between each other, and among people and devices.   And how much autonomy and leadership is afforded to the AI aspects.  Still not that common in practice.  Clear that automation has been here for a long time.  Starting with calculators, computers, then on to smartphones and tablets.    Creativity will be harder than repetitive tasks, it always has been.

AI will drive reskilling in problem solving, creativity and collaboration

A study from the Economist Intelligence Unit has found that executives do not believe that artificial intelligence will lead to job losses, but staff will need retraining  By Cliff Saran in ComputerWeek.,  Managing Editor
  
Uncertainty over security and data privacy represent workers’ main concerns over automation, a study from the Economist Intelligence Unit (EIU) has reported.

EIU’s Advance of automation report, based on a survey of 502 executives in Canada, France, Germany, India, Japan, Singapore, the UK and the US, found that just 9% of respondents said they were not using any automation.

More than half of the people surveyed (51%) said they made extensive use of automation, while 40% were moderate users of it, mainly for automating highly repetitive back-office functions.

However, the EIU found that more than a quarter of respondents (27%) expect automation to create opportunities for professional growth, and a similar number (26%) said they believe it could free up time for more human interaction. Another 37% said they believe automation would serve to increase employee engagement. ..... " 

Alexa Launches HIPAA Compliant Med Skills

Have now worked analyzing sever health oriented data intelligence interactions.   Now Amazon/Alexa makes move with skills.  Restrictions in their application are interesting. 

Amazon Alexa Launches First HIPAA-Compliant Medical Skills 
In TechCrunch  By Sarah Perez

Amazon has announced a program under which it is inviting select voice-app developers to create and launch healthcare skills for its Alexa virtual assistant that are compliant with the U.S. Health Insurance Portability and Accountability Act (HIPAA). Only select covered entities and business associates subject to HIPAA will be invited to the program, with Amazon itself providing the HIPAA-eligible environment for skill building. Accepted applicants will be allowed to use the Alexa Skills Kit, which currently supports skills that are able to transmit and receive protected health data. With the added skills, Alexa would be able to assist consumers in tasks like booking doctor appointments, accessing hospital post-discharge instructions, and checking on the status of prescription deliveries.  ... " 

Interaction Insights in Alexa Skills

This piece was published last year, but gave me some interesting insight recently about usage of skills.

Gain Interaction Insights Using New Analytics in the ASK Developer Console   By BJ Haberkorn

Today we added interaction path analysis to the Analytics tab on the Alexa Skills Kit (ASK) Developer Console. Interaction path analysis shows aggregate skill usage patterns in a visual format, including which intents your customers use, in what order. This enables you to verify if customers are using the skill as expected, and to identify interactions where customers become blocked or commonly exit the skill. You can use insights gained from interaction path analysis to make your flow more natural, fix errors, and address unmet customer needs.


(Note the Use of Sankey Diagrams)

View Interaction Paths over Multiple Time Intervals
As shown in the example below, interaction path analysis provides a visual representation of the flow of users from the invocation of your skill to subsequent intents. In this example, most customers moved from LaunchRequest to Intent1. A smaller segment invoked Intent2 instead. Interaction path analysis shows both custom intents and built-in intents, such as

Ask Developer Console for Interaction Insights

Just got a chance to look at this more closely.

Gain Interaction Insights Using New Analytics in the ASK Developer Console  By BJ Haberkorn

Today we added interaction path analysis to the Analytics tab on the Alexa Skills Kit (ASK) Developer Console. Interaction path analysis shows aggregate skill usage patterns in a visual format, including which intents your customers use, in what order. This enables you to verify if customers are using the skill as expected, and to identify interactions where customers become blocked or commonly exit the skill. You can use insights gained from interaction path analysis to make your flow more natural, fix errors, and address unmet customer needs.

View Interaction Paths over Multiple Time Intervals
As shown in the example below, interaction path analysis provides a visual representation of the flow of users from the invocation of your skill to subsequent intents. In this example, most customers moved from LaunchRequest to Intent1. A smaller segment invoked Intent2 instead. Interaction path analysis shows both custom intents and built-in intents, such as   ... "