Helping Alexa Assistant to Learn

Fascinating piece in the Alexa Dev blog on memory measures for Alexa Assistant, am examining.  See the full technical paper linked to below.  Lengthy

How Alexa May Learn to Retrieve Stored "Memories"

By Rasool Fakoor who is an applied scientist in the Alexa Intelligent Decisions group

In May 2018, Amazon launched Alexa’s Remember This feature, which enables customers to store “memories” (“Alexa, remember that I took Ben’s watch to the repair store”) and recall them later by asking open-ended questions (“Alexa, where is Ben’s watch?”). At this year’s IEEE Spoken Language Technologies conference, we presented a paper relating to the technology behind this feature.

Most memory retrieval services depend on machine learning systems trained on sample questions and answers. But they often suffer from the same problem: the machine learning systems are trained using one criterion of success — a loss function — but evaluated using a different criterion — the F1 score, which is a cumulative measure of false positives and false negatives.

In our paper, we use a reinforcement-learning-based model to directly train a memory retrieval system using the F1 score. While the model is not currently in production, our experiments show that it can deliver significant improvements in F1 score over methods that use other criteria during training.

Typically, a machine learning system is trained to minimize some loss function, which describes how far the system is from perfect accuracy. After every pass through the training data, a learning algorithm estimates the shape of the loss function and modifies the system’s settings, in an attempt to find a lower value for the function. It’s a process called gradient descent, because, essentially, the algorithm tries to determine which way the function slopes and to move down the slope. ... " 

Technical Paper    https://arxiv.org/pdf/1810.00679.pdf

"Direct Optimization of F-Measure for Retrieval-Based Personal Question Answering"