Learning, Forgetting and Reinforcement

 Very interesting thing,  came to our level of interest as we tried to analyze how consumers could 'forget' brands, and how that forgetting might be turned around.   A kind of learning and forgetting cycle.  Domain context is different, but applicable here?  An ultimate element of any kind of learning. 

Case study: Counteracting the forgetting curve with reinforcement technology

L&D practitioners know that learning cannot take root and impact sustained performance changes if it is once and done.

by Jeneane Becker,  February 6, 2023   in ChiefLearningOfficer

Employees are busier than ever. That’s why often in organizations, learning and development activities occur infrequently, typically during onboarding or annually for compliance purposes. But L&D practitioners know that learning cannot take root and impact sustained performance changes if it is once and done. The Ebbinghaus Curve (or forgetting curve) indicates that knowledge retention is a mere 10 percent only a week after completing training. 

The ultimate purpose of learning is to change behavior, which takes time, repetition, practice and continuous encouragement. Yet L&D teams often deliver a full suite of learning resources all at once. Learners consume the available training and perhaps the organization sees a spike in behavior change; however, without continued reinforcement over time, pre-training behaviors eventually return. That’s why the Choice University team asked ourselves how can we break the cycle of employees taking training periodically (i.e., when it’s “due”) and instead create a more regular habit of returning to the ChoiceU.com LMS over time, not just when they are initially onboarding or annually required.

Identifying the need

Recognizing the need for reinforcement methodology was a data-driven decision. An internal content analysis indicated steep drop-off with engagement of a learning asset 30 days post-launch. Alongside this data, external validation from Choice’s Business Intelligence team shows that hotels with higher Choice University engagement achieve stronger results with KPIs like revenue and guest satisfaction. Therefore, the need to drive “return” habits for learners matters more than simply LMS clicks, it makes a difference for hotels’ business. 

Additionally, hotel employee turnover rates are typically higher than other industries, so new learners are onboarding all the time at any of our 7,000-plus franchised properties worldwide. We estimate that, on average, for a learning asset launched six months ago, approximately 1,000 learners will have missed the asset at launch. If we do not reinforce and remember over time, key learning concepts may be completely missed by an increasing percentage of our learner population. 

Testing the concept

We piloted a blended solution for our customer loyalty program, one with which every hotel employee interacts. We developed “foundational” e-learning and video content followed by bite-sized “reinforcement” tools and resources to be strategically released over time, and then cyclically retired. Each iterative release of reinforcement content is intended to point back to or build upon a foundational learning asset, which drives new learners to take previously released content they might not have been aware of otherwise, while tenured learners refresh their knowledge and understanding.   ... ' 

AI Needs to Selectively Forget

We discovered his early on as we had to effectively maintain models.

Can AI Learn to Forget?     By Samuel Greengard  in the CACM

Communications of the ACM, April 2022, Vol. 65 No. 4, Pages 9-11   10.1145/3516514

Machine learning has emerged as a valuable tool for spotting patterns and trends that might otherwise escape humans. The technology, which can build elaborate models based on everything from personal preferences to facial recognition, is used widely to understand behavior, spot patterns and trends, and make informed predictions.

Yet for all the gains, there is also plenty of pain. A major problem associated with machine learning is that once an algorithm or model exists, expunging individual records or chunks of data is extraordinarily difficult. In most cases, it is necessary to retrain the entire model—sometimes with no assurance that that model will not continue to incorporate the suspect data in some way, says Gautam Kamath, an assistant professor in the David R. Cheriton School of Computer Science at the University of Waterloo in Canada.

The data in question may originate from system logs, images, health records, social media sites, customer relationship management (CRM) systems, legacy databases, and myriad other places. As right to be forgotten mandates appear, fueled by the European Union's General Data Privacy Regulation (GDPR) and the California Consumer Privacy Act (CCPA), organizations find themselves coping with potential minefields, including significant compliance penalties.

Not surprisingly, completely retraining models is an expensive and time-consuming process, one that may or may not address the underlying problem of making sensitive data disappear or become completely untraceable. What's more, there frequently is no way to demonstrate the retrained model has been fully corrected, and that it is entirely accurate and valid.

Enter machine unlearning. Using specialized techniques—including slicing databases into smaller chunks and adapting algorithms—it may be possible to induce selective 'amnesia' in machine learning models. The field is only beginning to take shape. "The goal is to find a way to rebuild models on the fly, rather than having to build an entirely new model every time the data changes," says Aaron Roth, a professor of computer and information science at the University of Pennsylvania.  ... ' 

IBM Uses Continual Learning

Forgetting in Neural Networks.  Intro is good, well worth understanding the topic, then becomes technical.

IBM Uses Continual Learning to Avoid The Amnesia Problem in Neural Networks

Tags: IBM, Learning, Neural Networks, Training  in KDNuggets

Using continual learning might avoid the famous catastrophic forgetting problem in neural networks.

SAS AI/ML Training

By Jesus Rodriguez, Intotheblock.

I often joke that neural networks suffers from a continuous amnesia problem in the sense that they every time they are retrained they lost the knowledge accumulated in previous iterations. Building neural networks that can learn incrementally without forgetting is one of the existential challenges facing the current generation of deep learning solutions. Over a year ago, researchers from IBM published a paper proposing a method for continual learning  proposing that allow the implementation of neural networks that can build incremental knowledge.

Neural networks have achieved impressive milestones in the last few years from beating Go to multi-player games. However, neural network architectures remain constrained to very specific domains and unable to transition its knowledge into new areas. Furthermore, current neural network models are only effective if trained over large stationary distributions of data and struggle when training over changing non-stationary distributions of data. In other words, neural networks can effectively solve many tasks when trained from scratch and continually sample from all tasks many times until training has converged. Meanwhile, they struggle when training incrementally if there is a time dependence to the data received. Paradoxically, most real world AI scenarios are based on incremental, and not stationary, knowledge. Throughout the history of artificial intelligence(AI), there have been several theories and proposed models to deal with the continual learning challenge.  ... " 

Learning New Ways to Continually Learn

Not quite I think of when I think of AGI (Artficial General Intelligence).    But sequences of useful tasks/learning can be seen as what humans do, provided they pay attention to both existing context and the changes in context introduced by the 'intelligence'. 

OpenAI’s Jeff Clune on deep learning’s Achilles’ heel and a faster path to AGI
 By Khari Johnson in Venturebeat

Neural networks learn differently from people. If a human comes back to a sport after years away, they might be rusty but they will still remember much of what they learned decades ago. A typical neural network, on the other hand, will forget the last thing it was trained to do. Virtually all neural networks today suffer from this “catastrophic forgetting.”

It’s the Achilles’ heel of machine learning, OpenAI research scientist Jeff Clune told VentureBeat, because it prevents machine learning systems from “continual learning,” the ability to remember previous tasks. But some systems can be taught to remember.

Before joining OpenAI last month to lead its multi-agent team, Clune worked with researchers from Uber AI Labs and the University of Vermont. This week, they collectively shared ANML (a neuromodulated meta-learning algorithm), which is able to learn 600 sequential tasks with minimal catastrophic forgetting.

“This is relatively unheard-of in machine learning. To my knowledge, it’s the longest sequence of tasks that AI has been able to do, and at the end of it, it’s still pretty good at all the tasks that it saw,” Clune said. “I think that these sorts of advances will be used in almost every situation where we use AI. It will just make AI better.”

Clune helped cofound Uber AI Labs in 2017, following the acquisition of Geometric Intelligence, and is one of seven coauthors of a paper called “Learning to Continually Learn” published Monday on arXiv.   https://arxiv.org/abs/2002.09571   ...... "

Neural Nets need to Learn to Forget

We were exposed to this and related issues when we first worked with neural methods. This is a big deal for all kinds of learning interactions.

Neural Networks Are Learning What to Remember and What to Forget  MIT Technology Review

MIT Technology Review
Deep learning is changing the way we use and think about machines. Current incarnations are better than humans at all kinds of tasks, from chess and Go to face recognition and object recognition.

But many aspects of machine learning lag vastly behind human performance. In particular, humans have the extraordinary ability to constantly update their memories with the most important knowledge while overwriting information that is no longer useful.

That’s an important skill. The world provides a never-ending source of data, much of which is irrelevant to the tricky business of survival, and most of which is impossible to store in a limited memory. So humans and other creatures have evolved ways to retain important skills while forgetting irrelevant ones.

The same cannot be said of machines. Any skill they learn is quickly overwritten, regardless of how important it is. There is currently no reliable mechanism they can use to prioritize these skills, deciding what to remember and what to forget.

Today that looks set to change thanks to the work of Rahaf Aljundi and pals at the University of Leuven in Belgium and at Facebook AI Research. These guys have shown that the approach biological systems use to learn, and to forget, can work with artificial neural networks too.  ... "

Can Font Influence Retention?

Quite a claim.  Would seem to be easily tested, as they mention ...

Can't Remember what you Read?  Blame the Font, not Forgetfulness.    In Wired,   by Arielle Pardes

Remember all those classics you devoured in comp-lit class? Neither do we. Research shows that we retain an embarrassingly small sliver of what we read. In an effort to help college students boost that percentage, a team made up of a designer, a psychologist, and a behavioral economist at Australia’s RMIT University recently introduced a new typeface, Sans Forgetica, that uses clever tricks to lodge information in your brain. The font-makers drew on the psychological theory of “desirable difficulty”—that is, we learn better when we actively overcome an obstruction. (It’s why flash cards create stronger neural connections in the brain and are a better method for recalling facts than passively studying notes.) Sans Forgetica is purposefully hard to decipher, forcing the reader to focus. One study found that students recalled 57 percent of what they read in Sans Forgetica, compared with 50 percent of the material in Arial, a significant difference. No word yet on the retention rate of Comic Sans.

Limiting the Right to be Forgotten

Been intrigued by the EU  'right to be forgotten'  laws a means to say you can have information about you scrubbed from the web.    Increasing your privacy at least in theory.    Seems only to be relevant only to big players who manage such information, like Google.    But unclear to me that other dark parts of the web are involved.   Google gets many such requests,  but approves only some.  Now new court rulings seem to say this is only relevant to the EU.   See below to this and links to more background information.

EU courts told 'right to be forgotten' law should not apply globally
The chief advisor at Europe's highest court has sided with Google. .... "

By Rachel England, @rachel_england in Engadget

The AI Need for Forgetting

Hackernoon discusses the need for forgetting.

Forgetting. A long time concept, and true in its essence, but forgetting needs to be in context, like remembering, and we do too little remembering of conversational context today,  Did lots of work where we looked at this in a maintenance interaction.   Need  to make sure that our systems still achieve the right measurable quality goal.   'Forgetting' implies random loss, say to just save space in memory, and it should not be.

Google Starts to Allow Forgetting

Have not heard about this anywhere else, but has some fascinating implications.  In the past we always assumed that knowledge would persist.  May that be a thing of the past from the point of view of the search engine and online history?   Already required by law in Europe, now moving to the US.   Article on this  In CWorld.  by Evan Schuman.
  

Seeking: Building AI that Builds AI

Building AI is hard work,  have been working on it for years.  Even if we are attempting to just improve relatively common cognitive business tasks.  Forgetting for now hoping to build complete 'brains'.   So can it be automated?   I ask further.  How should we look at the combination of supervised versus unsupervised training of such systems?  How should  we address the maintenance of such systems as their context evolves?   All big challenges.  In Wired, a look at how Google, Facebook, Microsoft and others are are looking  at this challenge to automation

Forgetting Organizations

An executive once told us as we sought to create expert systems: " ... You don't want to record and save my expertise, you want to forget it to make room for new creativity ... " .   Related, from K@W: Digital Transformation: Becoming a ‘Forgetting Organization 

Reshaping the Workspace With AI

Liked this  TechCrunch  piece because it addresses workspace and workflow in ways that AI claims usually do not. Not only 'What' will be done with cognitive, but 'How' it will be effectively used in day to day operations .... with what data ... and not forgetting the 'Why': ... faster, better, cheaper? All that leads to a better chance of it getting tried and installed and used.  A simple business process model, or even a concept diagram of the workspace are good places to start.

Forgetting Leads to Memory Holes in the Web

The EU rulings that you could ask for posts to be removed from Google web searches have led to more unusual and unexpected results.  Not that there are not already memory holes in the web. It seems that there are often external links in this blog that have gone bad.  Since items are so interconnected the The 'right to be forgotten', is often interlinked with useful information.  Watching the dynamics of this.

About Learning

Learning is a favorite topic of mine.  I supported a language laboratory study of sleep learning a long time ago and still follow this area.     Here in Wired "  ... an interview  with Robert Bjork, the director of the UCLA Learning and Forgetting Lab, a distinguished professor of psychology, and a massively renowned expert on packing things in your brain in a way that keeps them from leaking out. ,,, "

On the Age of the Informavore

In the Edge: ' We are apparently now in a situation where modern technology is changing the way people behave, people talk, people react, people think, and people remember. And you encounter this not only in a theoretical way, but when you meet people, when suddenly people start forgetting things, when suddenly people depend on their gadgets, and other stuff, to remember certain things... ' ... Some interesting takes, yes, I may rely on devices more, but I think as a result I am more informed than ever before.