The Risk and Uncertainty of it All

A space we played in early on, determining measures of uncertainty as we built AI and expert system based models.   Its part of any decision driving system,  That includes measures of uncertainty and risk.  Glad to see efforts to include it in machine learning systems today.  Such measures should lead to model improvements.  Now can these models also learn what aspects of the model design and data directly create problems of certainty?   

A Neural Network Learns When It Should Not Be Trusted    By MIT News

 

The advance could enhance safety and efficiency in artificial intelligence-assisted decision-making. 

Massachusetts Institute of Technology researchers have developed a way for deep learning neural networks to rapidly estimate confidence levels in their output.

Researchers at the Massachusetts Institute of Technology (MIT) and Harvard University have enabled a neural network to rapidly process data, yielding both predictions and confidence levels based on the quality of the available data.

This deep evidential regression technique, which estimates uncertainty from a single run of the neural network, could lead to safer results.

The team designed the network with bulked-up output, generating not only a decision but also a new probabilistic distribution capturing the evidence supporting that decision; these evidential distributions directly capture the model's confidence in its forecast.

Included is any uncertainty within the underlying input data and the model's final decision, which indicates whether uncertainty can be reduced by modifying the network itself, or whether the input data is merely noisy.

MIT's Daniela Rus said, "By estimating the uncertainty of a learned model, we also learn how much error to expect from the model, and what missing data could improve the model."

From MIT News

Confidence in Automated Systems

Interesting example of personal data being used in automated systems and how it is handled.

Confidence in automated systems  from Fraunhofer
Research News / 3.2.2020

When it comes to cars that drive themselves, most people are still hesitant. There are similar reservations with respect to onboard sensors gathering data on a driver’s current state of health. As part of the SECREDAS project, a research consortium including the Fraunhofer Institute for Experimental Software Engineering IESE is investigating the safety, security and privacy of these systems. The aim is to boost confidence in such technology.

A new system controls whether, and under what circumstances, personal data is allowed to be transferred to a specific destination.
© Fraunhofer IESE

A new system controls whether, and under what circumstances, personal data is allowed to be transferred to a specific destination.

There is still some way to go before people can be persuaded to embrace a new technology like self-driving cars. When it comes to taking decisions in road traffic, we tend to place greater trust in human drivers than in software. Boosting confidence in such connected, automated systems and their ability to meet safety and data privacy concerns – whether in the field of mobility or medicine: that’s the aim of the consortium behind the SECREDAS project. SECREDAS – which stands for “Product security for cross domain reliable dependable automated systems” – brings together 69 partners from 16 European countries, including the Fraunhofer Institute for Experimental Software Engineering IESE. This project is seeking to ensure that European OEMs remain competitive in this field. It has total funding of 51.6 million euros, with the EU contributing around 15 million euros to this sum.

Increasing the safety of self-driving cars

The control of autonomous vehicles lies to an ever greater extent in the hands of neural networks. These are used to assess everyday road-traffic situations: Is the traffic light red? Is another vehicle about to cross the road ahead? The problem with neural networks, however, is that it remains unclear just how they come to such decisions. “We’re therefore developing a safety supervisor. This will monitor in real time decisions taken by the neural network. If necessary, it can intervene on the basis of this assessment,” says Mohammed Naveed Akram from Fraunhofer IESE. “The safety supervisor uses classical algorithms, which focus on key parameters rather than assessing the overall situation – that’s what the neural networks do. Our work for the SECREDAS project is mainly about identifying suitable metrics for this purpose, but we are also looking at how best to take appropriate counter measures in order to avert danger.”   ... ." 

On AI Crisis of Confidence

More like a crisis of expectations.   Understanding is good, useful, needed.  But it is not the driving need.   The failures that will occur will first be those of results rather than those of understanding.  Its significant, just not enough to match the hype.  AI is still narrow and contextually insufficiently aware.

How AI will Address its Crisis of Confidence
By Brad Anderson  in ReadwriteWeb

Artificial intelligence has advanced by leaps and bounds in recent years, becoming smarter and more autonomous than thought possible. However, there’s one area where AI could use some improvement in the decade ahead: transparency. But how will AI address its crises of confidence?

Historically, AI has operated like a black box. Select developers knew how the algorithms inside worked, but for everyone else, the mechanics of the AI remained obscured. Businesses have asked users to trust that AI’s insights are complete and accurate. But without understanding where those insights originated — what data and logic informed their basis — it’s hard to trust AI is as intelligent as it’s made to be.  .... "

Understanding Classification Performance

And as long as you are doing any kind of machine learning,  you need to measure and understand its performance.  Since I am doing that now, I noted Jason Brownlee's piece noted below:

How to Report Classifier Performance with Confidence Intervals
by Jason Brownlee  in Machine Learning Process

Once you choose a machine learning algorithm for your classification problem, you need to report the performance of the model to stakeholders. ... This is important so that you can set the expectations for the model on new data. ... A common mistake is to report the classification accuracy of the model alone. .... "

Confidence vs Intelligence

Generalized thought, but useful.   A little like 'A for effort'.  Acquired skill is still very useful.  Backing up your confidence also useful.

Here Is Why Confidence Will Always Trump IQ
Having a high IQ is great, but confidence is a much more potent trait when it comes to success in the workplace and in life. .... "      By Ilya Pozon, Founder, Pluto.TV