Scientists Increasingly Cannot Explain How AI Works

Buying this?  No, not in general. 

Scientists Increasingly Cannot Explain How AI Works

By Vice, November 2, 2022

Though there is already a subset of AI known as Explainable AI (XAI), the general techniques it promotes are often diminutive and inaccurate.

Rarely do we ever question the basic decisions we make in our everyday lives, but if we did, we might realize that we cannot pinpoint the exact reasons for our preferences, emotions, and desires at any given moment. A similar problem exists in artificial intelligence (AI).

The people who develop AI are increasingly having problems explaining how it works and determining why it has the outputs it has. Deep neural networks often seem to mirror not just human intelligence but also human inexplicability.

From Vice

View Full Article  

On Investment in Explainable AI

Needs always to be explained in its usage and risk contexts.

AI Is Explaining Itself to Humans. It's Paying Off.   By Reuters, April 15, 2022, Comments

Startups and major technology companies are investing heavily in explainable artificial intelligence (XAI), as U.S. and EU regulators campaign for fairness and transparency in automated decision-making.  XAI advocates say it has helped make AI more effective in fields such as healthcare and sales.

Microsoft saw its LinkedIn subscription revenue increase 8% after providing its sales team with CrystalCandle software, which identifies clients potentially at risk of cancellation, while explaining its reasoning.

Skeptics say an AI’s explanations of why it made the predictions it did are still too unreliable. LinkedIn says an algorithm's integrity cannot be judged without understanding its reasoning, while tools like CrystalCandle, for example, could help physicians learn why AI predicts someone is at greater risk of disease.

From Reuters   

AI Talks Decisions and their Completeness.

Good approach.   There is I recall a way to rate how good an explanation of a decision is in healthcare.  How well was this done? Also, since should be a complete conversation, a give and take set of statements and questions, how well was that done?   Further, its important to note which questions were not asked, and how they relate to that complete 'explanation'.    Overall,  conversation correctness and completeness and how they relate to future decisions.  

AI Matches Cardiologists' Expertise, While Explaining Its Decisions

University of California, San Francisco News, Elizabeth Fernandez, August 26, 2021

Scientists at the University of California, San Francisco and the University of California, Berkeley designed an artificial intelligence (AI) algorithm that diagnosed cardiovascular ailments as well as expert cardiologists, while explaining its reasoning. The researchers trained the convolutional neural network on commonly accessible electrocardiogram (ECG) data. The researchers said the algorithm performed strongly across 38 different diagnoses in five broad diagnostic categories. Because the researchers incorporated "explainability" into the algorithm, it highlighted ECG segments critical for each diagnosis, which may boost physicians' confidence in using it. The researchers said their results “offer strong support for AI algorithms like neural networks to be incorporated into existing commercial ECG algorithms, since they perform better for many diagnoses, can improve over time and provide additional insights through explainability.”

Responsible and Explainable AI

I would call this more broadly, 'responsible decision making' to embrace what is called 'AI' and analytical decision making.  Useful and considered look at what this means. 

Responsible AI: Bridging From Ethics to Practice   By Ben Shneiderman  via CACM

Communications of the ACM, August 2021, Vol. 64 No. 8, Pages 32-35  10.1145/3445973

The high expectations of AI have triggered worldwide interest and concern, generating 400+ policy documents on responsible AI. Intense discussions over the ethical issues lay a helpful foundation, preparing researchers, managers, policy makers, and educators for constructive discussions that will lead to clear recommendations for building the reliable, safe, and trustworthy systems6 that will be commercial success. This Viewpoint focuses on four themes that lead to 15 recommendations for moving forward. The four themes combine AI thinking with human-centered User Experience Design (UXD).

Ethics and Design. Ethical discussions are a vital foundation, but raising the edifice of responsible AI requires design decisions to guide software engineering teams, business managers, industry leaders, and government policymakers. Ethical concerns are catalogued in the Berkman Klein Center report3 that offers ethical principles in eight categories: privacy, accountability, safety and security, transparency and explainability, fairness and non-discrimination, human control of technology, professional responsibility, and promotion of human values. These important ethical foundations can be strengthened with actionable design guidelines.

Autonomous Algorithms and Human Control. The recent CRA report2 on "Assured Autonomy" and the IEEE's influential report4 on "Ethically Aligned Design" are strongly devoted to "Autonomous and Intelligent Systems." The reports emphasize machine autonomy, which becomes safer when human control can be exercised to prevent damage. I share the desire for autonomy by way of elegant and efficient algorithms, while adding well-designed control panels for users and supervisors to ensure safer outcomes. Autonomous aerial drones become more effective as remotely piloted aircraft and NASA's Mars Rovers can make autonomous movements, but there is a whole control room of operators managing the larger picture of what is happening.

Humans in the Group; Computers in the Loop. While people are instinctively social, they benefit from well-designed computers. Some designers favor developing computers as collaborators, teammates, and partners, when adding control panels and status displays would make them comprehensible appliances. Machine and deep learning strategies will be more widely used if they are integrated in visual user interfaces, as they are in counterterrorism centers, financial trading rooms, and transportation or utility control centers.

Explainable AI (XAI) and Comprehensible AI (CAI). Many researchers from AI and HCI have turned to the problem of providing explanations of AI decisions, as required by the European General Data Protection Regulation (GDPR) stipulating a "right to explanation."13 Explanations of why mortgage applications or parole requests are rejected can include local or global descriptions, but a useful complementary approach is to prevent confusion and surprise by making comprehensible user interfaces that enable rapid interactive exploration of decision spaces.   ... '

Note in particular a look at 'trustworthy certification' proposed outlines for a number of large industries, to show how this might be applied by the nature of their operation ... '