Frontier in AI Training

We always need to consider the outlier case.      Its often in current and future training.... I like the idea of thinking about broadening the training to include 'nothing'.

The Next Frontier in AI: Nothing
How an overlooked feature of deep learning networks can turn into a major breakthrough for AI
By Max Versace in IEEE

This is a guest post. The views expressed here are solely those of the author and do not represent positions of IEEE Spectrum or the IEEE.

At an early age, as we take our first steps into the world of math and numbers, we learn that one apple plus another apple equals two apples. We learn to count real things. Only later are we introduced to a weird concept: zero… or the number of apples in an empty box.

The concept of “zero” revolutionized math after Hindu-Arabic scholars and then the Italian mathematician Fibonacci introduced it into our modern numbering system. While today we comfortably use zero in all our mathematical operations, the concept of “nothing” has yet to enter the realm of artificial intelligence.

In a sense, AI and deep learning still need to learn how to recognize and reason with nothing.

Is it an apple or a banana? Neither!
Traditionally, deep learning algorithms such as deep neural networks (DNNs) are trained in a supervised fashion to recognize specific classes of things.

In a typical task, a DNN might be trained to visually recognize a certain number of classes, say pictures of apples and bananas. Deep learning algorithms, when fed a good quantity and quality of data, are really good at coming up with precise, low error, confident classifications.

The problem arises when a third, unknown object appears in front of the DNN. If an unknown object that was not present in the training set is introduced, such as an orange, then the network will be forced to “guess” and classify the orange as the closest class that captures the unknown object—an apple!

Basically, the world for a DNN trained on apples and bananas is completely made of apples and bananas. It can’t conceive the whole fruit basket.....

Enter the world of nothing

While its usefulness is not immediately clear in all applications, the idea of “nothing” or a “class zero” is extremely useful in several ways when training and deploying a DNN.

During the training process, if a DNN has the ability to classify items as “apple,” “banana,” or “nothing,” the algorithm’s developers can determine if it hasn’t effectively learned to recognize a particular class. That said, if pictures of fruit continue to yield “nothing” responses, perhaps the developers need to add another “class” of fruit to identify, such as oranges.  .... "