A Google AI Model Developed an Unexpected Skill

ACM NEWS

A Google AI Model Developed an Unexpected Skill

Google CEO Sundar Pichai.

Google CEO Sundar Pichai said there are elements of how artificial intelligence systems learn and behave that still surprise experts.

Concerns about AI developing skills independently of its programmers' wishes have long absorbed scientists, ethicists, and science fiction writers. A recent interview with Google's executives may be adding to those worries.

In an interview on CBS's 60 Minutes on April 16, James Manyika, Google's SVP for technology and society, discussed how one of the company's AI systems taught itself Bengali, even though it wasn't trained to know the language. "We discovered that with very few amounts of prompting in Bengali, it can now translate all of Bengali," he said.

CEO Sundar Pichai confirmed that there are still elements of how AI systems learn and behave that still surprises experts: "There is an aspect of this which we call— all of us in the field call it as a 'black box'. You don't fully understand. And you can't quite tell why it said this." Pichai said the company has "some ideas" why this could be the case, but it needs more research to fully comprehend how it works.

CBS's Scott Pelley then questioned the reasoning for opening to the public a system that its own developers don't fully understand, but Pichai responded: "I don't think we fully understand how a human mind works either."

From Quartz

View Full Article    

Deep Learning is Overused

 Well put, always consider the simpler analysis that leads to better understanding first. 

Deep Learning Is Becoming Overused

Understanding the data is the first port of call

By Michael Grogan  in TowardsDataScience

There is always a danger when any model is used in a black-box fashion to analyse data, and models of the deep learning family are no exception.

Don’t get me wrong — there are certainly occasions where a model such as a neural network can outperform more simplistic models — but there are plenty of examples where this is not the case.

To use an analogy — suppose you need to buy a vehicle of some sort for transportation purposes. Buying a truck is a worthwhile investment if you regularly need to transport large items across long distances. However, it is a blatant waste of money if you simply need to go to the local supermarket to pick up some milk. A car (or even a bicycle if you are climate-conscious) is sufficient to carry out the task in question.

Deep learning is starting to be used in the same way. We are starting to simply feed these models with the relevant data, assuming that performance will surpass that of simpler models. Moreover, this is often done without properly understanding the data in question; i.e. recognising that deep learning would not be necessary if one had an intuitive grasp of the data. ... ' 

Video: Interpretable Machine Learning

I mentioned this article in an earlier post, where I discuss in more detail, here here a short video introduction.

Techniques for Interpretable Machine Learning from CACM on Vimeo.

Mengnan Du and Xia Hu discuss "Techniques for Interpretable Machine Learning," a Review Article in the January 2020 CACM. ... 

Techniques for Interpretable Machine Learning

Very good piece I am reading in the January CACM.  The most important aspect of considering AI-ML type models in the real world.  Good introduction, useful key insights, but ultimately quite technical.  Bottom line is that research is still needed and 'Model explanation and surprising artifacts are often two sides of the same coin'.  Complex models may extract and codify biases and other 'artifacts' of metadata from training data.   Test and re-test under varying context.  Maintenance is more that just tracking performance over time.  Consider embedded models of risk.

I highly recommend subscribing to CACM if you are technically involved.

Techniques for Interpretable Machine Learning
By Mengnan Du, Ninghao Liu, Xia Hu

Communications of the ACM, January 2020, Vol. 63 No. 1, Pages 68-77
10.1145/3359786

Machine learning is progressing at an astounding rate, powered by complex models such as ensemble models and deep neural networks (DNNs). These models have a wide range of real-world applications, such as movie recommendations of Netflix, neural machine translation of Google, and speech recognition of Amazon Alexa. Despite the successes, machine learning has its own limitations and drawbacks. The most significant one is the lack of transparency behind their behaviors, which leaves users with little understanding of how particular decisions are made by these models. Consider, for instance, an advanced self-driving car equipped with various machine learning algorithms does not brake or decelerate when confronting a stopped firetruck. This unexpected behavior may frustrate and confuse users, making them wonder why. Even worse, the wrong decisions could cause severe consequences if the car is driving at highway speeds and might ultimately crash into the firetruck. The concerns about the black-box nature of complex models have hampered their further applications in our society, especially in those critical decision-making domains like self-driving cars.

Interpretable machine learning would be an effective tool to mitigate these problems. It gives machine learning models the ability to explain or to present their behaviors in understandable terms to humans,10 which is called interpretability or explainability and we use them interchangeably in this article. Interpretability would be an indispensable part for machine learning models in order to better serve human beings and bring benefits to society. For end users, explanation will increase their trust and encourage them to adopt machine learning systems. From the perspective of machine learning system developers and researchers, the provided explanation can help them better understand the problem, the data and why a model might fail, and eventually increase the system safety. Thus, there is a growing interest among the academic and industrial community in interpreting machine learning models and gaining insights into their working mechanisms.

Interpretable machine learning techniques can generally be grouped into two categories: intrinsic interpretability and post-hoc interpretability, depending on the time when the interpretability is obtained.23 Intrinsic interpretability is achieved by constructing self-explanatory models which incorporate interpretability directly to their structures. The family of this category includes decision tree, rule-based model, linear model, attention model, and so on. In contrast, the post-hoc one requires creating a second model to provide explanations for an existing model. The main difference between these two groups lies in the trade-off between model accuracy and explanation fidelity. Inherently interpretable models could provide accurate and undistorted explanation but may sacrifice prediction performance to some extent. The post-hoc ones are limited in their approximate nature while keeping the underlying model accuracy intact.  ... "

Google DeepMind Links ID with Decision Process

Quite an interesting claim.   The use of deep learning methods to identify problems using data and then applying process embedded solutions.   Here in the area of medicine:  Diagnosing 3D retinal scans.  The method being more transparent than simple deep learning methods.   And much closer to addressing process models and applications.  Will this solve the 'black box' (non transparency) problem of neural AI?  To be seen, but I like the idea.

Google DeepMind might have just solved the “Black Box” problem in medical AI

Deep Mind’s study published last week in Nature Medicine, presenting their Artificial Intelligence (AI) product capable of diagnosing 50 ophthalmic conditions from 3D retinal OCT scans. Its performance is on par with the best retinal specialists and superior to some human experts.

This AI product’s accuracy and range of diagnoses are certainly impressive. It is also the first AI model to reach expert level performance with 3D diagnostic scans. From a clinical point-of-view, however, what is even more groundbreaking is the ingenious way in which this AI system operates and mimics the real-life clinical decision process. It addresses the “Black Box” issue which has been one of the biggest barriers to the integration of AI technologies in healthcare.

DeepMind’s AI system addressed the “Black Box” by creating a framework with two separate neural networks. Instead of training one single neural network to identify pathologies from medical images, which would require a lot of labelled data per pathology, their framework decouples the process into two: 1) Segmentation (identify structures on the images) 2) Classification (analyze the segmentation and come up with diagnoses and referral suggestions)  .... "

Feedback and Blacker Boxes

Thoughts on the topic of complexity and understanding the operational specifics of what we have done.

The Blacker the Box  By Michael Kaminsky

There has been a lot of discussion in the data science community about the use of black-box models, and there is lots of really fascinating ongoing research into methods, algorithms, and tools to help data scientists better introspect their models. While those discussions and that research are important, in this post I discuss the macro-framework I use for evaluating how black the box can be for a prediction product.

In this post I do not get into the weeds of complexity penalization algorithms or even how to weigh the tech debt associated with additional complexity. Instead, I want to take a step back and discuss how I think about “prediction” problems at a more macro level, and how I value accuracy and complexity for different types of problems.

The thesis of this post is:

The faster the feedback on prediction accuracy, the blacker the box can be. The slower the feedback, the more your models should be explicit and formal.

In this post I talk through some examples of fast feedback problems and what makes them more amenable to black-box prediction algorithms (provided you have the proper infrastructure) as well as slower feedback problems and how one might approach predictions in those situations.

Fast Feedback

The machine learning community spends the bulk of its time working on and talking about fast feedback problems. Problems with fast feedback are defined by 1) the ability to quickly evaluate the correctness of a prediction1 and 2) the ability to play the game near infinite amounts of time2. Some examples of fast feedback problems are:

Chess: it is easy to verify which player has won or lost. Feedback takes only as long as the length of the game.
Conversion for an Ad Placement: Feedback to Google or Facebook on whether you clicked a given advertisement, and whether you subsequently converted  3 is nearly instantaneous.
Movie Recommendations: For a given list of potential movies to watch, Netflix gets near instantaneous feedback when you do or do not watch some of the content they have elevated for you. .... "

Myth of the Machine Learning Black Box

Was unable to get to see this, but of interest.  Most all machine learning projects need to be updated,  maintained and reevaluated.   Maybe they are not black boxes .... but are effectively so, because the decision makers have to do a considerable amount of work to figure them out.

The Myth of the Machine Learning Black Box
Added by Tim Matteson on June 21, 2017 

Critics describe machine learning as a "black box," where data goes in and a prediction comes out, without visibility into how the prediction was derived. This lack of transparency makes it difficult to evaluate and update predictive models as conditions change or new sources of data become available. But today's machine learning systems are not black boxes, allowing data scientists and business professionals alike to understand how a model makes its predictions.

In this Data Science Central webinar, DataRobot will discuss how today's automated machine learning systems provide the information and visualizations that deliver deep insights that break out of the black box.

Speaker: Greg Michaelson, Director of DataRobot Labs -- DataRobot

Hosted by: Bill Vorhies, Editorial Director -- Data Science Central

Judging the Algorithm

Excellent piece.    I have been involved in several projects which look at the correctness of algorithmic methods.   One point to make where a distinction is necessary.   A Black Box model implies the details of how and/or why a model works are unknown or are being hidden.  An algorithm usually implies the how/why of the model is known.  But its precise parameters may or may not be revealed.    'Black Box' implies fewer details are being revealed, and usually makes me worry more about the stability of the method being used.

The topic explored in this article below will become more crucial.  Algorithms need to be judged correct and compared for specific contexts.

The great question of the 21st century: Whose black box do you trust?
Algorithms shape choice not just for consumers but for businesses.
By Tim O'Reilly September 15, 2016

The role of technology-fueled algorithms in shaping our society, and how to use them responsibly, is one of the topics I'll be exploring at the Next:Economy Summit in San Francisco, October 10-11. Join me there.

Some years ago, John Mattison, the chief medical information officer of Kaiser Permanente, the large integrated health provider, said to me, "The great question of the 21st century is going to be 'Whose black box do you trust?'" Mattison was talking about the growing importance of algorithms in medicine, but his point, more broadly, was that we increasingly place our trust in systems whose methods for making decisions we do not understand. (A black box, by definition, is a system whose inputs and outputs are known, but the system by which one is transformed to the other is unknown.)

A lot of attention has been paid to the role of algorithms in shaping the experience of consumers. Much less attention has been paid to the role of algorithms in shaping the incentives for business decision-making. .... "