Robots Learning from Videos

 Researchers Expand Ability of Robots to Learn from Videos

Carnegie Mellon University School of Computer Science

Aaron Aupperlee

June 20, 2023

A model developed by Carnegie Mellon University (CMU) researchers allows robots to learn how to complete household tasks by watching videos of people doing chores. With the Vision-Robotics Bridge (VRB) model, two robots learned a dozen chores, such as opening drawers and taking pots off the stove, through observation. VRB builds on a previous model, In-the-Wild Human Imitating Robot Learning (WHIRL). With VRB, robots can learn a new task from a video in just 25 minutes. The model employs the concept of affordances (perceived potential actions), in which the robot determines contact points and direction of movement in the performance of a particular task. Said CMU's Shikhar Bahl, "This work could enable robots to learn from the vast amount of Internet and YouTube videos available."  ... ' 

Duolingo and AI

There may be some hints at using/planning assistant skills here. 

How Duolingo’s AI Learns What You Need to Learn

500+IEEE Spectrum by Burr Settles / February 05, 2023

It’s lunchtime when your phone pings you with a green owl who cheerily reminds you to “Keep Duo Happy!” It’s a nudge from Duolingo, the popular language-learning app, whose algorithms know you’re most likely to do your 5 minutes of Spanish practice at this time of day. The app chooses its notification words based on what has worked for you in the past and the specifics of your recent achievements, adding a dash of attention-catching novelty. When you open the app, the lesson that’s queued up is calibrated for your skill level, and it includes a review of some words and concepts you flubbed during your last session.

Duolingo, with its gamelike approach and cast of bright cartoon characters, presents a simple user interface to guide learners through a curriculum that leads to language proficiency, or even fluency. But behind the scenes, sophisticated artificial-intelligence (AI) systems are at work. One system in particular, called Birdbrain, is continuously improving the learner’s experience with algorithms based on decades of research in educational psychology, combined with recent advances in machine learning. But from the learner’s perspective, it simply feels as though the green owl is getting better and better at personalizing lessons.

The three of us have been intimately involved in creating and improving Birdbrain, of which Duolingo recently launched its second version. We see our work at Duolingo as furthering the company’s overall mission to “develop the best education in the world and make it universally available.” The AI systems we continue to refine are necessary to scale the learning experience beyond the more than 50 million active learners who currently complete about 1 billion exercises per day on the platform.

Although Duolingo is known as a language-learning app, the company’s ambitions go further. We recently launched apps covering childhood literacy and third-grade mathematics, and these expansions are just the beginning. We hope that anyone who wants help with academic learning will one day be able to turn to the friendly green owl in their pocket who hoots at them, “Ready for your daily lesson?”

The origins of Duolingo .. 

Learning, Forgetting and Reinforcement

 Very interesting thing,  came to our level of interest as we tried to analyze how consumers could 'forget' brands, and how that forgetting might be turned around.   A kind of learning and forgetting cycle.  Domain context is different, but applicable here?  An ultimate element of any kind of learning. 

Case study: Counteracting the forgetting curve with reinforcement technology

L&D practitioners know that learning cannot take root and impact sustained performance changes if it is once and done.

by Jeneane Becker,  February 6, 2023   in ChiefLearningOfficer

Employees are busier than ever. That’s why often in organizations, learning and development activities occur infrequently, typically during onboarding or annually for compliance purposes. But L&D practitioners know that learning cannot take root and impact sustained performance changes if it is once and done. The Ebbinghaus Curve (or forgetting curve) indicates that knowledge retention is a mere 10 percent only a week after completing training. 

The ultimate purpose of learning is to change behavior, which takes time, repetition, practice and continuous encouragement. Yet L&D teams often deliver a full suite of learning resources all at once. Learners consume the available training and perhaps the organization sees a spike in behavior change; however, without continued reinforcement over time, pre-training behaviors eventually return. That’s why the Choice University team asked ourselves how can we break the cycle of employees taking training periodically (i.e., when it’s “due”) and instead create a more regular habit of returning to the ChoiceU.com LMS over time, not just when they are initially onboarding or annually required.

Identifying the need

Recognizing the need for reinforcement methodology was a data-driven decision. An internal content analysis indicated steep drop-off with engagement of a learning asset 30 days post-launch. Alongside this data, external validation from Choice’s Business Intelligence team shows that hotels with higher Choice University engagement achieve stronger results with KPIs like revenue and guest satisfaction. Therefore, the need to drive “return” habits for learners matters more than simply LMS clicks, it makes a difference for hotels’ business. 

Additionally, hotel employee turnover rates are typically higher than other industries, so new learners are onboarding all the time at any of our 7,000-plus franchised properties worldwide. We estimate that, on average, for a learning asset launched six months ago, approximately 1,000 learners will have missed the asset at launch. If we do not reinforce and remember over time, key learning concepts may be completely missed by an increasing percentage of our learner population. 

Testing the concept

We piloted a blended solution for our customer loyalty program, one with which every hotel employee interacts. We developed “foundational” e-learning and video content followed by bite-sized “reinforcement” tools and resources to be strategically released over time, and then cyclically retired. Each iterative release of reinforcement content is intended to point back to or build upon a foundational learning asset, which drives new learners to take previously released content they might not have been aware of otherwise, while tenured learners refresh their knowledge and understanding.   ... ' 

Alarmed by AI Chatbots, Universities Start Revamping How They Teach

 And the concerns will build.

 Alarmed by A.I. Chatbots, Universities Start Revamping How They Teach  By The New York Times, January 17, 2023

Colleges and universities have been reluctant to ban the new chatbot because administrators doubt the move would be effective.

While grading essays for his world religions course last month, Antony Aumann, a professor of philosophy at Northern Michigan University, read what he said was easily "the best paper in the class." It explored the morality of burqa bans with clean paragraphs, fitting examples and rigorous arguments.

A red flag instantly went up.

Mr. Aumann confronted his student over whether he had written the essay himself. The student confessed to using ChatGPT, a chatbot that delivers information, explains concepts and generates ideas in simple sentences — and, in this case, had written the paper.

From The New York Times

Recent Game playing example

 

Games can be useful to test learning and alternative process options.

Why Researchers Are Teaching AI to Play Minecraft    By Popular Science, December 2, 2022

Nuclear power plant model made in Minecraft An artist's approximation of a nuclear power plant model made in Minecraft.  Using Video Pre-Training (VPT),the new AI program could construct items in Minecraft previously unattainable to bots reliant only on reinforcement learning.

OpenAI has developed a Minecraft-playing bot that can build pixelated tools and buildings in the game that require more than 20,000 consecutive actions via a combination of imitation and reinforcement learning. The bot, trained on 70,000 hours of human gameplay, is the first to build "diamond tools," which take human players 20 minutes and 24,000 actions, on average, to construct.

Imitation learning requires each step to be hand-labeled, but the researchers used a separate neural network to handle labeling via Video Pre-Training....  The researchers said the use of imitation and reinforcement learning in combination could pave the way for advancements in self-driving vehicles and nuclear fusion research.

From Popular Science    

View Full Article

More: Is AI Becoming Sentient?

I say no, even the term itself is flawed, 

ACM NEWS

Is AI Becoming Sentient?

By Gregory Goth, Commissioned by CACM Staff, October 27, 2022

Insights into human cognition need to become more granular in order to accurately evaluate AI’s capabilities – or to accurately compare them to those of humans, said Konstantinos Voudouris of the U.K.'s University of Cambridge.

There has been more than a little sensational speculation in recent months regarding the ability of artificial intelligence (AI) to attain sentience, but Bruce McNaughton, distinguished professor of neurobiology and behavior at the University of California at Irvine, does not waste much time pondering the possibility.

In fact, McNaughton doesn't ascribe much to the idea of sentience, period. He discourages his students from using the term, as well as "consciousness."

"I discourage them from using the term because as a scientist, I think of the brain as a physical system that obeys the laws of physics," McNaughton said. "We just didn't understand that implementation of the laws of physics could become so incredibly complex through the laws of evolution, and I think most people have only 

Konstantinos Voudouris, a psychologist and graduate student researcher at the Leverhulme Centre for the Future of Intelligence of the U.K.'s University of Cambridge, shares McNaughton's sentiments to a fair degree. "It's beguiling almost to be anthropomorphic about how these systems are behaving," said Voudouris, first author of a recent study that directly compared the cognitive abilities of AI agents and children age 6-10. "It's almost a quality of human psychology to anthropomorphize things, but the psychologist can come in and scientifically evaluate that hypothesis against the many alternatives that exist."

Neither McNaughton nor Voudouris are computer scientists or engineers, yet their recent work on artificial intelligence is emblematic of a surge in multidisciplinary research; the mechanics of human cognition, which have served as the theoretical underpinnings of AI development for decades, are receiving greater attention in the development of AI systems that are subject to ever-greater expectations of what they will be able and need to do — and vice versa.

McNaughton said there is more back-and-forth between how cognitive science and artificial intelligence can affect each other because the principles, the mathematics, and the fundamental approaches of machine learning are the same principles and problems the brain confronts: how to make efficient generalizable knowledge that can be flexible and used in different situations.

"In order to capture the statistical structure of the world as we experience it or as an artificial network experiences it, it takes many, many trials in order to gain a good statistical representation of the domain of the data," he said. "The brain has that problem, and artificial neural networks have that problem."

For instance, he said, cognitive scientists are still searching for how the brain actually does the equivalent of backpropagation if, indeed, it doesn't do backpropagation itself: "That is a central problem in computational neuroscience. But that's where neuroscientists who have some ability to follow the machine learning literature can gain insight from that field."

Jay McClelland, director of Stanford University's Center for Mind, Brain, Computation, and Technology, who has published numerous influential explorations of cognitive science and AI with McNaughton, said he also sees a burgeoning dialogue between machine learning scholars and cognitive scientists.

"We do have a two-way street, in the sense that work from AI is at least leading to ways in which people in neuroscience can see how they can engage in the discussion with other people in other fields," he said. "And computer science can offer hypotheses and alternative ways of thinking about exactly what the brain is doing and how it is solving problems, or raise questions we need to answer as brain scientists."

Lifelong learning in machines and humans

One of the major factors driving this computer science/cognitive science dialogue is the Lifelong Learning Machines (L2M) research program launched by the U.S. Defense Advanced Research Projects Agency (DARPA) in 2017, with University of Massachusetts AI expert Hava Siegelmann as the project's first program manager.

The L2M program's core goal was to create AI systems that could take new data, leverage previously learned information, and learn on the fly. Traditional AI architectures, Siegelmann said, fall far short of that. "If you train a network to separate cats and dogs, then you use the same network to separate elephants from tigers, if you use just regularizers, your system won't be able to separate elephants from cats, because it was never a task that it learned."

Siegelmann convened a cross-discipline pool of computer scientists, neuroscientists, biologists, and others to fundamentally change the depth to which research into machine and human cognition interacted.

"They really stepped outside the box and tried to incorporate a range of ideas and thinking in the field," McNaughton said, "and part of that was a subset of neuroscientists who were also interested in these problems. Suffice to say that connection has been strengthened, at least from the perspective of the interests of the machine learning community in neuroscience."

"I wanted the biologists and neuroscientists to tell me the mechanism of how learning works in the brain," Siegelmann said. "I didn't want them to just tell me it goes from the hippocampus to the cortex. We know that. I wanted them to give me a mechanism in such detail I could actually write equations and program them."

The latest research by McNaughton's lab, published in the Proceedings of the National Academy of Science, hewed closely to Siegelmann's stipulations by addressing a persistent problem in artificial neural networks. Termed catastrophic interference or catastrophic forgetting, it is the rapid loss of previously acquired knowledge if new information is introduced too quickly, essentially because the new information re-weights the network to an extent that the system virtually forgets what it has previously learned. Traditionally, artificial network architectures try to alleviate this by re-introducing everything the system has learned as new information is introduced, but this approach becomes both time- and compute-resource impractical, especially if a system is expected to function successfully on the fly.

McNaughton's group, led by the study's first author, graduate student Rajat Saxena, refined a learning system introduced by McNaughton and McClelland in 2020 called Similarity Weighted Interleaved Learning (SWIL). The SWIL theory suggests that learning in artificial networks can be made more efficient by introducing only a subset of old items that share substantial representational similarity with the new information: "By using such similarity-weighted interleaved learning, artificial neural networks can learn new information rapidly with a similar accuracy level and minimal interference, while using a much smaller number of old items presented per epoch," the group concluded.

In their original paper, McNaughton, McClelland, and Andrew Lampinen concluded SWIL performed similarly to networks that interleaved every old item with the new ones to be learned, but used 40% fewer items. They did not find, though, that it scaled beyond a simple neural network.

The latest paper successfully scaled SWIL to work on traditional classification datasets (Fashion-MNIST, CIFAR10, and CIFAR100) as well or better than existing schemes such as Fully Interleaved Learning (FIL), Focused Learning (FoL), and Equally Weighted Interleaved Learning (EqWIL). The team concluded that SWIL's future, at least in terms of AI, probably lays in complementing other learning techniques, such as generative replay or elastic weight consolidation. And, while McNaughton called the latest SWIL research the "evolution" of a breakthrough concept rather than a breakthrough in itself, he did say it stimulated questions about human cognition.  .... ' 

AI Teaching Cursive Handwriting

Unusual application for these times, but I agree it can have useful side effects.

Applied AI Teaches Handwriting    By Esther Shein

Communications of the ACM, October 2022, Vol. 65 No. 10, Pages 19-20   10.1145/3554919

Researchers from Germany's Karlsruhe Institute of Technology (KIT) and pen-maker Stabilo are collaborating on an artificial intelligence (AI)-based pen to teach schoolchildren what is becoming a lost art in an increasingly digital world: handwriting.

The joint project—Kaligo-based Intelligent Handwriting Teacher (KIHT)—is funded by the German Federal Ministry of Education and Research.

German children are taught to write by redrawing the shape of letters, which requires them to think about writing, explains Tanja Harbaum, a researcher at KIT who is involved with the project. "We want them to be able to write without having to think about writing. That's what we as adults do."

The eyes of unskilled writers are not able to keep up with writing, and "that's really a problem because if you force a child to redraw shapes, they won't be able to practice fluent writing at the same time," according to Harbaum.

While teaching shapes should be the first step, children are "painting the letters, not writing them," says Peter Kämpf, head of special product development at Stabilo. "Painting means that pen movement is slow and deliberate, with close hand-eye coordination. Therefore, the next step must focus on the dynamics of writing," which is the wrist movement, he says, and not focus on shapes "until the writing movement has developed to the point where it is an overlearned motion that does not depend on optical control."

This not only speeds up the writing process but also frees up cognitive capacity, he says.

Styli have been shown to enhance the ability to write. "Writing with the finger is more suitable for performing large, but not very accurate motions, while writing with the stylus leads to a higher precision and more isotropic motion performance," according to a 2015 study published in the National Library of Medicine.

Regardless of whether a stylus or an old-fashioned pen or pencil is used, however, studies have found there is a significant connection between handwriting, cognitive development, and the ability to retain information.

In 2015, Finland became one of the first countries to phase out handwriting instruction altogether, to keep pace with technological progress. (Although U.S. schools are not required to teach cursive writing, schools in some states continue to do so.) Some researchers do not agree with Finland's decision.

University of Washington professor Virginia Berninger told the online news site Qcostarica that writing with a pen not only helps develop fingers, but also thinking skills, because the brain works harder to write.

Other studies support the fact that handwriting is a complex task that requires more brainpower to process a word than just reading or typing it. Handwriting is both physical and mental, and the brain has to apply motor skills and thought processing when applying pen to paper to create words. 

Even adults can benefit from continuing to use their handwriting skills. A 2021 study by Johns Hopkins University published in the peer-reviewed journal Psychological Science posits practicing handwriting "refines fine-tuned motor skills and creates a perceptual-motor experience that appears to help adults learn generalized literacy-related skills surprisingly faster and significantly better than if they tried to learn the same material by typing on a keyboard or watching videos."

"Our results clearly show that handwriting compared with nonmotor practice produces faster learning and greater generalization to untrained tasks than previously reported," researchers Robert Wiley and Brenda Rapp told Psychology Today. "Furthermore, only handwriting practice leads to the learning of both motor and amodal symbolic letter representations."  ..... ' 

Fake Data for Faster Learning

Embedded generalizations?

'Fake' Data Helps Robots Learn the Ropes Faster

University of Michigan News, June 29, 2022

University of Michigan (U-M) researchers expanded training datasets for teaching robots to work with soft objects like ropes and fabrics, to expedite the learning process. U-M's Dmitry Berenson and Peter Mitrano augmented an optimization algorithm to enable a computer to make human-level generalizations, forecasting how dynamics observed in one case might recur in others and generating variants of the first experiment's result that serve the robot in the same way. In simulations, the expanded dataset improved the success rate of a robot looping a rope around an engine block by 48%, which increased to 70% after training. An experiment using a physical robot to perform the same task almost doubled its success rate over the course of 30 tries. .... 

Video Games Might Make Kids Smarter?

Not the games I have seen them play, here some interesting results.

Boomer Humor Was Wrong: Video Games Might Make Kids Smarter in Extremetech

By Jessica Hall on May 25, 2022 at 3:13 pm

“TV will rot your brain!” We’ve all heard that old canard. The idea that screen time makes kids dumber is a staple of the “Father, I cannot click the book” genre of boomer humor. But science is turning that narrative on its head. A newly published report using data from an ABCD Study indicates that screen time doesn’t rot kids’ brains after all. On the contrary: video games might actually make kids smarter.

The ABCD Study

The Adolescent Brain and Cognitive Development (ABCD) study is a gigantic longitudinal study of American child health and brain development.

The ABCD study timeline.

Participants begin at 9-10 years of age. Through its massive slate of tests and surveys, the project records a wide array of behavioral, biometric, and genetic information from participants and their parents. Then, project scientists follow up with participant families until the kids are 19-20 years old.

Data from the project is freely available for other researchers to use in their own studies. Once or twice a year, the ABCD Study releases another updated dataset. Now, researchers from Vrije Universiteit Amsterdam and the Karolinska Institutet of Sweden have used that data to find out what video gaming really does to childrens’ brains.

Video Games Can Make Kids Smarter

“For our study,” two of the new study’s authors explain in a joint statement, “we were specifically interested in the effect of screen time on intelligence – the ability to learn effectively, think rationally, understand complex ideas, and adapt to new situations.”

In particular, their model looked at how much time kids spent staring at glowing rectangles, and even how they used their screen time. For example, most of the kids in the study used their screen time in three different ways: watching videos (e.g. YouTube), socializing online, or playing video games. It compared gamers and non-gamers on tasks including reading comprehension, memory, visual-spatial processing, and executive function.

The researchers wanted to cover a wide variety of subdomains of intelligence. “However, intelligence is highly heritable in the populations we’ve studied so far,” study author Dr. Bruno Sauce explained to us over Zoom. Furthermore, “genetic and socioeconomic factors were our two major confounders.” So, to account for variations in these factors, the researchers rolled in genetic data and socioeconomic information from the participants’ parents. .... ' 

Using Synthetic Data

Synthetic data training from real data.

When It Comes to AI, Can We Ditch the Datasets?

MIT News, Adam Zewe, March 15, 2022

Massachusetts Institute of Technology (MIT) researchers have demonstrated the use of a generative machine-learning model to produce synthetic data, based on real data, to train another model for image classification. Researchers showed the generative model millions of images containing objects in a specific class, after which it learned those objects' appearance in order to generate similar objects. MIT's Ali Jahanian said generative models also learn how to transform underlying training data, and connecting a pretrained generative model to a contrastive learning model enabled both models to work together automatically. The results show that a contrastive representation learning model trained only on synthetic data can learn visual representations that rival or top those learned from real data. In analyzing how the number of samples influenced the model's performance, researchers determined that, in some cases, generating larger numbers of unique samples facilitated additional enhancements.  ... ' 

Brain Cells Learn Pong

Seems quite remarkable, if it is what I think it is, how far could be be extended, repeated?    Is it ethical to use human brain cells for this?

ACM TECHNEWS

Human Induced Cells Grown in Petri Dish Learn to Play Pong Faster Than AI

By DailyMail.com, December 30, 2021

Researchers at Cortical Labs in Australia demonstrated that human-induced neurons grown in a petri dish can be taught to play the retro videogame Pong in only five minutes.

The DishBrain system is made of brain cells grown on microelectrode arrays that can stimulate the cells. The researchers sent electrical signals either to the right or left of the array to indicate the video game ball's location, and the brain cells would fire neurons to move the paddle accordingly.

DishBrain learned the game in 10 to 15 rallies, gaming sessions that last for 15 minutes, but it takes 5,000 rallies for an artificial intelligence to learn the game.

"Using this DishBrain system, we have demonstrated that a single layer of in vitro cortical neurons can self-organize and display intelligent and sentient behavior when embodied in a simulated game-world," the researchers said in their published report.

From DailyMail.com  Full Article

Joking Chatbots Help Learning?

 Brought the below up before, here is more detail. Humor is powerful, but how do we harness it? Need a good way.  Thoughts?  See my Humor link.

Joke-Cracking Chatbots Boost Learning Levels

By Paul Marks, Commissioned by CACM Staff, July 20, 2021

Technology has brought us many wonderful things, but chatbots are not one of them. On banking and e-commerce sites, for instance, where these text-based conversational agents have been pressed into service to replace customer-support staff, even simple requests are often met with baffling arrays of options.

For example, a bank's online chatbot recently asked me which of four types of savings account I was interested in – but it did not explain how they differed from each other. When I typed in "I don't know" the bankbot replied tersely: "That is not an option". It then looped me back to those original options. No wonder some consumer commentators – like this one at Forbes – say such clumsy chatbot implementations are "killing customer service".

So, when I heard researchers in Canada had decided to add a sense of humor to chatbots, I feared the worst. Surely making light of giving people inadequate information would be adding insult to injury? Well, apparently not: the researchers have found that comedy-capable chatbots could have an important role, although in education – not customer support.

Speaking at the virtual Conference on Human Factors in Computing Systems (CHI2021) in May, human-computer interaction researchers led by Jessy Ceha and Ken Jen Lee of the University of Waterloo in Ontario, Canada, described how their team set out to investigate whether a particular way of studying a subject, called learning-by-teaching, might be enhanced by a witty chatbot.

In garden variety learning-by-teaching, typically three students research a subject using certain teacher-approved resources like books, Websites, diagrams, and photographs. They then prepare a lesson and teach that subject to another small group of students, the process of lesson-planning having helped them not only improve their own factual recall of the subject at hand, but also, in having to work out how to teach the subject, develop their problem-solving skills.

The Waterloo team wondered, what if the teaching group got to coach a smart chatbot instead of fellow students? Further, they wondered whether a chatbot with a sense of humor, in getting a few laughs out of its teachers, might better motivate them, lower their stress and anxiety levels, and help them learn more?  ... ' 

Brain Like System Mimics Learning

Hmm, not sure of this, but interesting

 'Brain-Like Device' Mimics Human Learning in Major Computing Breakthrough

The Independent (U.K.), Anthony Cuthbertson, April 30, 2021

A device modeled after the human brain by researchers at Northwestern University and the University of Hong Kong can learn by association, via synaptic transistors that simultaneously process and store information. The researchers programmed the circuit to associate light with pressure by pulsing a light-emitting diode (LED) lightbulb and then applying pressure with a finger press. The organic electrochemical material enabled the device to construct memories, and after five training cycles it associated light with pressure and could detect pressure from light alone. Northwestern's Jonathan Rivnay said, "Because it is compatible with biological environments, the device can directly interface with living tissue, which is critical for next-generation bioelectronics."

Learning from Big Mistakes

How about linking it to after action reviews?   Move closer to the process and the results.

How to Learn from the Big Mistake You Almost Make  by Kristen Senz

A brush with disaster can lead to important innovations, but only if employees have the psychological safety to reflect on these close calls, says research by Amy C. Edmondson, Olivia Jung, and colleagues.

What if businesses could learn from their worst mistakes without actually making them? How might the same progress and innovation occur, without firms incurring the costs associated with such errors?

The results of a recent study about close calls in health care suggest that when people feel secure about speaking up at work, incidents in which catastrophe is narrowly averted rise to the surface, spurring important growth and systems improvement.

“People don't pay enough attention, especially in the business world, to the potential goldmine of near-misses,” says Harvard Business School Professor Amy C. Edmondson, who studies psychological safety and organizational learning.

Incidents that almost result in loss or harm often pass unnoticed, in part because workers worry about being associated with vulnerability or failure. But when leaders frame near misses as free learning opportunities and express the value of resilience to their teams, the likelihood that workers will report such incidents increases.

That was the main finding of Resilience vs. Vulnerability: Psychological Safety and Reporting of Near Misses with Varying Proximity to Harm in Radiation Oncology, a study by Edmondson, the Novartis Professor of Leadership and Management at Harvard Business School, and Olivia Jung, a doctoral student at HBS. Co-authors on the paper, which was published in The Joint Commission Journal on Quality and Patient Safety, included UCLA physicians Palak Kundu, John Hegde, Michael Steinberg, and Ann Raldow, and medical physicist Nzhde Agazaryan.  ... "

Wal-Mart Embraces Immersive VR Learning

Good piece with useful details of the effort.   Consider the volume of training required.   Was involved with some sales training efforts, but this takes it quite further.  Our innovation centers also allowed us to stage sales interactions with actual consumers.  And with the CEO of our company as well.    Did some experimentation with VR, but that was still too immature for the typical consumer to use. 

Case study: Walmart embraces immersive learning

Virtual reality is revolutionizing the way the retail giant’s associates learn.

By  Sarah Fister Gale   In Chief Learning Officer

In 2016, Walmart had an emerging issue among its learning programs. The $4 trillion retailer has 1.5 million workers in the U.S., and most of them needed training on how to handle complex customer situations — specifically, training that wouldn’t be disruptive to the customer experience.

“We can’t do that in the store,” says Kate Kressen, senior manager II of learning content and development for Walmart in Bentonville, Ark. “And it’s very hard to recreate a live store environment in a training program.”

For a long time they relied on classroom instructors giving lectures and quizzes, or static online courses that associates clicked through on their own. But neither format could convey the heightened experience of dealing with certain situations in the flow of work.

“We can talk all day, but until you understand the tension that associates and managers feel, it doesn’t really translate,” Kressen says.

Armchair coach

Around that same time, Derek Belch was launching Strivr, a Palo Alto, Calif.-based athletic training company that uses virtual reality as an immersion tool to give athletes a way to practice their craft off the field. Belch had previously been an assistant football coach at Stanford while writing his master’s thesis on using VR to train football players. The project was so impressive that Stanford’s head coach provided Belch with funding to launch Strivr, which he co-founded with Stanford VR professor Jeremy Bailenson.

“When we started Strivr, we were only focused on the sports world,” Belch says. But about a year after launching, he got a surprising call from Brock McKeel, senior director of digital operations at Walmart, who’d seen Strivr’s VR software being used for quarterback training. “He wanted to talk about employee training,” Belch says.

By that time, Strivr had worked with more than 30 NFL and college teams, but Belch had never considered the potential of using his immersive training technology to teach store employees.

However, as he and his team discussed the opportunity, it started to make sense. “As I talked to Walmart about how they train their employees, and what they needed them to learn, we realized that our formula for athletes wasn’t that different,” he says. His developers wouldn’t need to reinvent the software, they just needed to reengineer the experience for a store environment. “It turned out to be a little easier than we expected,” he says. “Stores are a lot more static than a football field.”

Belch was convinced he could create a course that would work for Walmart, though the potential scale of the project was daunting. Walmart wanted Strivr to create programs that could be run in all 200 learning academies, which are training centers attached to larger Walmart stories. And eventually, the retailer wanted to roll it out to 4,000 locations. “It was important that Strivr be able to scale their solution to meet our needs,” Kressen says, “because we needed to get a handle on training 1.2 million associates.”  ... '

BERT for Unsupervised Training

 A nice, fairly compact, somewhat technical view of how to use BERT for unsupervised training problems.    And some unexpected uses.  True, everyone should know how to use this method.  Click through for full detail. 

For unsupervised task solving ... 

BERT is a prize addition to the practitioner’s toolbox   By Ajit Rajasekharan  in TowardsDataScience

Figure 1. Few reasons why BERT is a valuable addition to a practitioner’s toolbox apart from its well known use of fine-tuning for downstream tasks. (1) BERT’s learned vocabulary of vectors (in say 768 dimensional space) serve as targets that masked output vectors predict and learn from prediction errors during training. After training, these moving targets settle into landmarks that can be clustered and annotated (a one-time step) and used for classifying model output vectors in a variety of tasks — NER, relation extraction etc. (2) A model pre-trained enough to achieve a low next sentence prediction loss (in addition to the masked word prediction loss) yields quality CLS vectors representing any input term/phrase/sentence.  ... ' 

Detecting Object Permanence and more

Reinforcement learning problem. Learning like children do.

AI Agents Learned Object Permanence by Playing Hide and Seek

By IEEE Spectrum  February 17, 2021

Researchers at the Allen Institute for AI (AI2) demonstrated that artificial intelligence agents learned the concept of object permanence — that objects hidden from view are still there — by playing hide and seek.

The agents, playing as both hiders and seekers, learned the game "Cache" via reinforcement learning. The agents began learning about the environment by taking random actions, like pulling on drawers, and dropping objects in random places. Their game play improved as they learned from outcomes, with the hider, for instance, learning that it had selected a good hiding place when the seeker failed to find the object.

Subsequent testing showed that the agents understood the principles of containment and object permanence and were able to rank images based on how much free space they contained.

The agents performed as well or better than models trained on the gold-standard ImageNet.

From IEEE Spectrum

AI and Tacit Knowledge

Useful, mostly non-technical paper on the topic.  We learned much about this when we trained with explicit rules.  How do we effectively leverage the statement:  'we know more than we can tell"?

Polanyi's Revenge and AI's New Romance with Tacit Knowledge  By Subbarao Kambhampati    Communications of the ACM, February 2021, Vol. 64 No. 2, Pages 31-32  10.1145/3446369

In his 2019 Turing Award Lecture, Geoff Hinton talks about two approaches to make computers intelligent. One he dubs—tongue firmly in cheek—"Intelligent Design" (or giving task-specific knowledge to the computers) and the other, his favored one, "Learning" where we only provide examples to the computers and let them learn. Hinton's not-so-subtle message is that the "deep learning revolution" shows the only true way is the second.

Hinton is of course reinforcing the AI Zeitgeist, if only in a doctrinal form. Artificial intelligence technology has captured popular imagination of late, thanks in large part to the impressive feats in perceptual intelligence—including learning to recognize images, voice, and rudimentary language—and bringing fruits of those advances to everyone via their smartphones and personal digital accessories. Most of these advances did indeed come from "learning" approaches, but it is important to understand the advances have come in spheres of knowledge that are "tacit"—although we can recognize faces and objects, we have no way of articulating this knowledge explicitly. The "intelligent design" approach fails for these tasks because we really do not have conscious theories for such tacit knowledge tasks. But, what of tasks and domains—especially those we designed—for which we do have explicit knowledge? Is it forbidden to give that knowledge to AI systems?

(Robot Image)  "Human, grant me the serenity to accept the things I cannot learn, data to learn the things I can, and wisdom to know the difference."

The polymath Polanyi bemoaned the paradoxical fact that human civilization focuses on acquiring and codifying "explicit" knowledge, even though a significant part of human knowledge is "tacit" and cannot be exchanged through explicit verbal instructions. His "we can know more than we can tell" dictum has often been seen as a pithy summary of the main stumbling block for early AI efforts especially in perception.

Polanyi's paradox explains to a certain extent why AI systems wound up developing in a direction that is almost the reverse of the way human babies do. Babies demonstrate aspects of perceptual intelligence (recognizing faces, voices and words), physical manipulation (of putting everything into their mouths), emotional intelligence, and social intelligence, long before they show signs of expertise in cognitive tasks requiring reasoning skills. In contrast, AI systems have demonstrated reasoning abilities—be they expert systems or chess—long before they were able to show any competence in the other tacit facets of intelligence including perception.

In a sense, AI went from getting computers to do tasks for which we (humans) have explicit knowledge, to getting computers to learn to do tasks for which we only have tacit knowledge. The recent revolution in perceptual intelligence happened only after labeled data (such as cats, faces, voices, text corpora, and so forth) became plentiful, thanks to the Internet and the World Wide Web, allowing machines to look for patterns when humans are not quite able to give them explicit know-how. .... '

Neural Networks are Bayesian

 Just reading this, worth thinking about. Technical.

Neural networks are fundamentally Bayesian  in TowardsDataScience By Chris Mingard

Stochastic Gradient Descent approximates Bayesian sampling

Deep neural networks (DNNs) have been extraordinarily successful in many different situations — from image recognition and playing chess to driving cars and making medical diagnoses. However, in spite of this success, a good theoretical understanding of why they generalise (learn) so well is still lacking.

In this post, we summarise results from three papers, which provide a candidate for a theory of generalisation in DNNs [1,2,3].     .... "

Learning Better with Shape-Shifting

 Like the example of modifying objects to address specific elements of training.   Unusual application, uses outside of sports?

Better Learning with Shape-Shifting Objects

MIT News, Adam Conner-Simons

Shape-shifting objects that can help users improve their skills is an area of investigation for Massachusetts Institute of Technology (MIT) researchers, who conceived of a basketball hoop that trains players more effectively by shrinking and raising to help them make shots more consistently. Experiments demonstrated that training on the auto-adaptive hoop improved player performance more than using a static hoop or the manually-adaptive mode. Autodesk's Fraser Anderson said, "You don't have to rely on your own sense of whether or not you've mastered a skill: the system can do that and take out the self-doubt, overconfidence, or guesswork."   ... '