Simulation and Causation Leading to Machine Learning

Hmm quite a thought,   Want to see this broadly adapted and usefully applied.  Technical supporting papers.

CausalCity: Introducing a high-fidelity simulation with agency for advancing causal reasoning in machine learning

Published June 29, 2021

By Daniel McDuff , Principal Researcher  Yale Song , Senior Researcher  Sai Vemprala , Senior Researcher  Vibhav Vineet , Senior Researcher  Shuang Ma , Senior Researcher  Ashish Kapoor , Partner Research Manager

The ability to reason about causality, and ask “what would happen if…?’’ is one property that sets human intelligence apart from artificial intelligence. Modern AI algorithms perform well on clearly defined pattern recognition tasks but fall short generalizing in the ways that human intelligence can. This often leads to unsatisfactory results on tasks that require extrapolation from training examples, e.g., recognizing events or objects in contexts that are different from the training set. To address this problem, we have built a high-fidelity simulation environment, called CausalCity, which is designed for developing algorithms that improve causal discovery and counterfactual reasoning of AI.

To understand the problem better, imagine if we developed a self-driving car confined to the streets of a neighborhood in Arizona with few pedestrians, wide, flat roads and street signs with English writing. If we deployed the car on the narrow, busy streets of Delhi, where street signs are written in Hindi, pattern recognition would be insufficient to operate safely. The pattern in our “training set’’ would be very different from our deployment context. Yet, somehow humans can adapt so quickly to situations that they haven’t previously observed that someone with an Arizona state-issued driving license is allowed to drive a car in India.

In our recent paper, “CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning“, we take a closer look at this problem and propose a new high-fidelity simulation environment.  ... ' 

Can AI Understand Cause and Effect?

Key point is made,  in human conversations we continually make cause-effect assumptions and consideration.

What AI still can’t do in TechnologyReview
Artificial intelligence won’t be very smart if computers don’t grasp cause and effect. That’s something even humans have trouble with.

by Brian Bergstein

In less than a decade, computers have become extremely good at diagnosing diseases, translating languages, and transcribing speech. They can outplay humans at complicated strategy games, create photorealistic images, and suggest useful replies to your emails.

Yet despite these impressive achievements, artificial intelligence has glaring weaknesses.

Machine-learning systems can be duped or confounded by situations they haven’t seen before. A self-driving car gets flummoxed by a scenario that a human driver could handle easily. An AI system laboriously trained to carry out one task (identifying cats, say) has to be taught all over again to do something else (identifying dogs). In the process, it’s liable to lose some of the expertise it had in the original task. Computer scientists call this problem “catastrophic forgetting.”

These shortcomings have something in common: they exist because AI systems don’t understand causation. They see that some events are associated with other events, but they don’t ascertain which things directly make other things happen. It’s as if you knew that the presence of clouds made rain likelier, but you didn’t know clouds caused rain. .... "

Could We Forgive a Machine?

Still at the edge of real life considerations.  It is really still an exploration.  But even in the last era of AI, we looked at aspects of fault and liability.   We built some simple causal models to help management understand how this might look.  The lawyers and regulators will have to decide. 

Could we forgive a machine? Study explores forgiveness in the context of robotics and AI   by Ingrid Fadelli , Tech Xplore

As more robots make their way into society, it is important to consider the ethical and moral implications of having them complete tasks that can have a significant impact on people's lives. If robots and machines are to become widely used in situations where they could seriously affect human lives, for instance by driving cars or giving elderly people the medication they need on a daily basis, developers should first consider the associated implications.

With this in mind, Michael Nagenborg, a researcher at the University of Twente in the Netherlands, has recently carried out a study investigating what could happen if a robot or machine makes a terrible mistake that has detrimental consequences for its users. His paper, published in Springer Link's Technology, Anthropology, and Dimensions of Responsibility journal, specifically explores the question of whether humans would be able to forgive a robot if it does something wrong, from a philosophical standpoint. ...  "

More information: Michael Nagenborg. Can We Forgive a Robot?, Technology, Anthropology, and Dimensions of Responsibility (2020).   DOI.org/10.1007/978-3-476-04896-7_11

Causal Monitoring for Distributed Systems: IOT Solutions

Spent some time looking at monitoring and troubleshooting systems, with the obvious need to infer causation.   Here an intro to 'Pivot Tracing', a means of causal monitoring I had not heard of.    Would seem to be very useful for interconnected IOT.   Technical.

Pivot Tracing: Dynamic Causal Monitoring for Distributed Systems
By Jonathan Mace, Ryan Roelke, Rodrigo Fonseca
Communications of the ACM, March 2020, Vol. 63 No. 3, Pages 94-102
10.1145/3378933

Monitoring and troubleshooting distributed systems are notoriously difficult; potential problems are complex, varied, and unpredictable. The monitoring and diagnosis tools commonly used today—logs, counters, and metrics—have two important limitations: what gets recorded is defined a priori, and the information is recorded in a component- or machine-centric way, making it extremely hard to correlate events that cross these boundaries. This paper presents Pivot Tracing, a monitoring framework for distributed systems that addresses both limitations by combining dynamic instrumentation with a novel relational operator: the happened-before join. Pivot Tracing gives users, at runtime, the ability to define arbitrary metrics at one point of the system, while being able to select, filter, and group by events meaningful at other parts of the system, even when crossing component or machine boundaries. Pivot Tracing does not correlate cross-component events using expensive global aggregations, nor does it perform offline analysis. Instead, Pivot Tracing directly correlates events as they happen by piggybacking metadata alongside requests as they execute. This gives Pivot Tracing low runtime overhead—less than 1% for many cross-component monitoring queries.  ... "

Causation to Provide the Why

Basic causation is a great start.   Why did this happen? We are doing it all the time.  Its one of our basic knowledge processing capabilities that lead to learning.    We can figure out the answer by observation and combing observations to build rules of operation.   Or we can be taught specific rules, or even  imprecise rules of thumb, to help us process knowledge.   Combining things we have observed or not.   They must include things like causation, space and time relationships.  It is this kind of knowledge we need to do general AI.  Not just more data.   Its more than just unstructured data.  Its about combining all learning experiences we experience into an interacting data rich architecture that we can use.   Like the direction of the below:

An AI Pioneer Wants His Algorithms to Understand the 'Why'
  Will Knight, in Wired

Yoshua Bengio, a researcher at the University of Montreal in Canada who is co-recipient of the 2018 ACM A.M. Turing Award for contributions to the development of deep learning, thinks artificial intelligence will not realize its full potential until it can move beyond pattern recognition and learn more about cause and effect, which would make existing AI systems smarter and more efficient. A robot that understands dropping things causes them to break, for example, would not need to toss dozens of vases onto the floor to see what happens to them. Bengio is developing a version of deep learning that can recognize simple cause-and-effect relationships. His team used a dataset that maps causal relationships between real-world phenomena in terms of probabilities. The resulting algorithm essentially forms a hypothesis about which variables are causally related, and then tests how changes to different variables fit the theory.  ... " 

Data, Data Science, Causal Thinking

"The Seven Tools of Causal Inference, with Reflections on Machine Learning," by ACM  A.M. Turing Award recipient Judea Pearl, describes tools that overcome obstacles to human-level machine intelligence. Pearl delivers a message to machine-learning and AI experts in an original video at bit.ly/2GUEyJW.

Excerpt from the long paper, ultimately positioning our challenge:

Key insights:

- Data Science is a two-body problem, connecting data and reality, including the forces behind the data.

- Data Science is the art of interpreting reality in the light of data, not a mirror through which data sees itself from different angles.

- The ladder of causation is the double helix of causal thinking, defining what can and cannot be learned about actions and about worlds that could have been. ...   "

Judea Pearl on Causal Inference

 Back to the need for better integrated inference.   Its not enough to just find patterns,  we need to insert them in our cognitive work. 

"The Seven Tools of Causal Inference, with Reflections on Machine Learning," by ACM A.M. Turing Award recipient Judea Pearl, describes tools that overcome obstacles to human-level machine intelligence. Pearl delivers a message to machine-learning and AI experts in an original video at bit.ly/2GUEyJW.

Full paper in the Communications of the ACM    Technical, but contains excellent overview pieces that are essential to understand the future of AI beyond Deep Learning.   And my point made above.

Ultimately this makes the case of connecting any kind of analytics (like machine learning) to human augmentation and interaction.

Judea Pearl on Causality Tools and Machine Learning

  Causality scientist Judea Pearl looks at causality and some of the limitations of machine learning systems.  Technical paper, but useful scan for practitioners.

The seven tools of causal inference with reflections on machine learning

The seven tools of causal inference with reflections on machine learning Pearl, CACM 2018

With thanks to @osmandros for sending me a link to this paper on twitter.

In this technical report Judea Pearl reflects on some of the limitations of machine learning systems that are based solely on statistical interpretation of data. To understand why? and to answer what if? questions, we need some kind of a causal model. In the social sciences and especially epidemiology, a transformative mathematical framework called ‘Structural Causal Models’ (SCM) has seen widespread adoption. Pearl presents seven example tasks which the model can handle, but which are out of reach for associational machine learning systems.  ... "

Causality and Data Science

Of interest, reviewing.

Causal Data Science  By Adam Kelleher
Physicist; Data @ BuzzFeed; Adjunct Prof. at Columbia

I started a series of posts aimed at helping people learn about causality in data science (and science in general), and wanted to compile them all together here in a living index. This list will grow as I post more:  ... " 

Math of Causation

Not to say we humans do this well either,  but if we hope to have machines do it we have to consider how.

Solution: ‘The Slippery Math of Causation’

The all-too-intuitive picture of a straight arrow going from cause to effect is far too simplistic to describe the real world.   By Pradeep Mutalik in Quanta

Our latest Insights puzzle attempted to model multifactorial causation with problems that involved three causal factors whose different types of interactions either produced or did not produce an effect. The goal was to challenge the all-too-intuitive picture of a straight arrow going from cause to effect as far too simplistic to describe the real world. In fact, a recent Quanta article by Veronique Greenwood describes the omnigenic model of complex traits, with the startling self-explanatory title “Theory Suggests That All Genes Affect Every Complex Trait.” The thinking that inspired our puzzle and is manifest in the omnigenic theory was captured visually by Paul Laurienti. In the diagram below, think of the genes as the causes and the complex traits as the effects, and you will see that it fits the new theory perfectly. Thank you, Paul!  .... "

Determining the Cause of War

In my early days at the Pentagon, working for the Joint Chiefs, we built and ran conflict simulations. But we were less interested in causes than we were in outcomes.  At the time we would have said the causes are too many, too complex, too subtle to control even for a simulation.

" ... DARPA, BAE Systems and the Air Force Research Lab are working to pioneer new computer simulations, algorithms and advanced software to provide military decision makers with organized, near real-time information on causes of war and conflict in operational scenarios. .... "

Can DARPA & BAE-Developed Computer Algorithms Determine the True Causes of War?   An emerging computer simulation analyzes multiple factors to determine the cause of war

By Kris Osborn - Warrior Maven

DARPA, BAE Systems and the Air Force Research Lab are working to pioneer new computer simulations, algorithms and advanced software to provide military decision makers with organized, near real-time information on causes of war and conflict in operational scenarios.

Drawing upon a range of otherwise disconnected sources of raw data, the new software program is designed to use reasoning algorithms and simulations to analyze intelligence reports, academic theories, environmental factors and details from operational scenarios and other kinds of user input.

“It is about taking information from disparate sources which would be impossible for a person to consume in a short amount of time,” Jonathan Goldstein, Senior Principal Scientist, Autonomy Controls and Estimation, BAE Systems, told Warrior Maven in an interview.

The Air Force Research Laboratory recently awarded a $4.2 million deal to BAE Systems to develop CONTEXT; DARPA is sponsoring BAE's efforts.

The emerging product, called Causal Exploration of Complex Operational Environments (CONTEXT) models different political, territorial and economic tensions that often cause conflict. These nodes, or variables making up a complex, yet interwoven tapestry of causes, include things like economic tensions, terrorism, tribal or religious conflict and issues about resources or territorial disputes – among other things.

“The technology evaluates causal insertions in different forms and innovates them into a model of interwoven causal relationships present in otherwise disconnected sources. We are building a model that can rapidly be used by an expert, so that when a new conflict flares up, decision-makers can understand the underlying issues,” Goldstein said.

While on the surface, organizing and performing some analytics of large pools of data might bring AI to mind, CONTEXT evaluates material input by users and does not necessarily access massive volumes of historical or stored data. Nonetheless, it does appear to perform some measure of automation and AI like functions, in so far as it organizes and integrates different sources for a human decision maker.

“This shortens the decision cycle. People are not good at maintaining a causal model with complexity in their head. The software creates a large graph of causes, evaluates approaches and examines the potential consequences of a given approach,” Goldstein explained.

Automation and AI, which are of course progressing at near lighting speed these days, are often described in terms of easing the “cognitive burden,” meaning they can quickly perform analytics and a range of procedural functions to present to a human operating in a command control capacity.

At the same time, causes of conflict are often a complex byproduct of a range of more subjectively determined variables – impacted by concepts, personalities, individual psychology, historical nuances and larger sociological phenomena. This naturally raises the question as to how much even the most advanced computer programs could account for these and other somewhat less “tangible” factors.

Leading AI and cybersecurity experts often say that advanced computer algorithms can analyze data and quickly perform procedural functions far more quickly than human cognition – yet there are nonetheless still many things which are known to be unique to human cognition. Humans solve problems, interpret emotions and at times respond to certain variables in a way that the best computer technology cannot.

“War causation is always over determined. Even with advanced statistical regressions on extremely large data sets, it is unlikely that what causes conflict can be determined with accuracy,” Ross Rustici, Senior Director, Intelligence Services, Cybereason – and former DoD Cyber Lead Intrusion Analyst and Technical Lead for DoD, East Asia, told Warrior Maven.  .... "

Addressing Causation vs Curve Fitting

The below comes from Inference.vc   It addresses some of the comments by Judea Pearl's recent post commented on here.  Beyond the first few paragraphs it is thoughtful but very technical.

ML beyond Curve Fitting: An Intro to Causal Inference and do-Calculus 

You might have come across Judea Pearl's new book, and a related interview which was widely shared in my social bubble. In the interview, Pearl dismisses most of what we do in ML as curve fitting. While I believe that's an overstatement (conveniently ignores RL for example), it's a nice reminder that most productive debates are often triggered by controversial or outright arrogant comments. Calling machine learning alchemy was a great recent example. After reading the article, I decided to look into his famous do-calculus and the topic causal inference once again.

Again, because this happened to me semi-periodically. I first learned do-calculus in a (very unpopular but advanced) undergraduate course Bayesian networks. Since then, I have re-encountered it every 2-3 years in various contexts, but somehow it never really struck a chord. I always just thought "this stuff is difficult and/or impractical" and eventually forgot about it and moved on. I never realized how fundamental this stuff was, until now.

This time around, I think I fully grasped the significance of causal reasoning and I turned into a full-on believer. I know I'm late to the game but I almost think it's basic hygiene for people working with data and conditional probabilities to understand the basics of this toolkit, and I feel embarrassed for completely ignoring this throughout my career.

In this post I'll try to explain the basics, and convince you why you should think about this, too. If you work on deep learning, that's an even better reason to understand this. Pearl's comments may be unhelpful if interpreted as contrasting deep learning with causal inference. Rather, you should interpret it as highlighting causal inference as a huge, relatively underexplored, application of deep learning. Don't get discouraged by causal diagrams looking a lot like Bayesian networks (not a coincidence seeing they were both pioneered by Pearl) they don't compete with, they complement deep learning. .... " 

Judea Pearl Criticizes Machine Learning

Quite interesting view.  Pearl's view is interesting.  Bayesian networks in particular has shown a more broadly insightful and transparent view to modeling than machine learning.    But machine learning deep learning can target narrower problems more specifically.

How a Pioneer of Machine Learning Became One of Its Sharpest Critics
Judea Pearl helped artificial intelligence gain a strong grasp on probability, but laments that it still can't compute cause and effect.

 By Kevin Hartnett in The Atlantic

Artificial intelligence owes a lot of its smarts to Judea Pearl. In the 1980s he led efforts that allowed machines to reason probabilistically. Now he’s one of the field’s sharpest critics. In his latest book, The Book of Why: The New Science of Cause and Effect, he argues that artificial intelligence has been handicapped by an incomplete understanding of what intelligence really is.

Three decades ago, a prime challenge in artificial-intelligence research was to program machines to associate a potential cause to a set of observable conditions. Pearl figured out how to do that using a scheme called Bayesian networks. Bayesian networks made it practical for machines to say that, given a patient who returned from Africa with a fever and body aches, the most likely explanation was malaria. In 2011 Pearl won the Turing Award, computer science’s highest honor, in large part for this work.

But as Pearl sees it, the field of AI got mired in probabilistic associations. These days, headlines tout the latest breakthroughs in machine learning and neural networks. We read about computers that can master ancient games and drive cars. Pearl is underwhelmed. As he sees it, the state of the art in artificial intelligence today is merely a souped-up version of what machines could already do a generation ago: find hidden regularities in a large set of data. “All the impressive achievements of deep learning amount to just curve fitting,” he said recently.  .... " 

Quantum, Cause and Effect

Will Quantum computing be able to take advantage of the very strange nature of time in Quantum physics?   Causality?  At the micro level perhaps, but beyond?    Introduction to the idea in Nature.

How Quantum Trickery Can Scramble Cause and Effect    In Nature

“Most of the attempts to understand quantum mechanics involve trying to save some aspects of the old classical picture, such as particle trajectories,” says Brukner. But history shows us that what is generally needed in such cases is something more, he says — something that goes beyond the old ideas, such as a new way of thinking about causality itself. “When you have a radical theory, to understand it you usually need something even more radical.”  ... " 

We still live in a macro world, which we must model.  Quantum lives in the microstructure.  Bridging these will be only the first challenge.

Why: A Guide to Finding and Using Causes

I brought this book up some time ago.    And as I read it am struck by how essential is is to understand cause and effect when working with data, analytics, machine learning and assistant services.  This is MORE important than understanding code or data structure.

The book does an excellent job, starting non-technically, and extending to explain recent research. It is worth at least a thoughtful scan.  If you don't understand this, your modeling can be dangerously invalid.   The link below goes to a free 23 page sample pdf of the book. After that, buy it for reference.

Why: A Guide to Finding and Using Causes
by Samantha Kleinberg

"Kleinberg expertly guides readers on a tour of the key concepts and methods for identifying causal relationships, with a clear and practical approach that makes Why unlike any other book on the subject. Accessible yet comprehensive, Why is essential reading for scientific novices, seasoned experts, and anyone else looking to learn more from data."  .... '

Samantha Kleinberg's Site.  and book ordering information.

Myths About Machine Learning

A number of useful points are embedded here.  Useful to know these objections, since they will be brought up by thoughtful management.  These are high level observations.  I am not sure I would call all of these 'myths'.  They all have elements of caution and truth to them, but the author also suggests under what contexts they might be true or not.   In the real world you are always under some assumptions for any kind of useful modeling, so its useful to know what they are.   Machine Learning Myths.

InsightETE Seeks Root Cause in Process

Just brought to my attention, and applicable to all kinds of process measuring and management.  Whats more important than determining root cause in a process?    Digging deeper, join me.

Welcome To InsightETE
Founded in 1999, InsightETE is the brain child of founder and systems engineer Bill Johns.

InsightETE is the first and only company in the world to offer a proprietary and patented method (PATENT No.: US 7,577,701 B1) to perform root cause analysis in 15 minutes or less.

InsightETE’s software gives you the ability to measure and troubleshoot IT system performance on a granular level. No other company in the world can truthfully claim the same ability.

InsightETE clients can measure true response time, track service levels, and reduce outages as they root out problems from their verified source. What’s more, they see an increase in their customer service satisfaction by eliminating service level disagreements.  ... " 

When and Why of Causation

Adam kelleher
Physics PhD; Principal Data Scientist at BuzzFeed  : Thoughtful Medium piece:

" ... So what is causality good for? Anytime you decide to take an action, in a business context or otherwise, you’re making some assumptions about how the world operates. That is, you’re making assumptions about the causal effects of possible actions. Most of the time, we only consider very simple actions: I should eat, because the causal effect of “eating” on “hunger” is negative. I should show more impressions of this ad, because the causal effect of impressions on pageviews is positive. What about more complex interventions? What about the downstream effects? Maybe showing more impressions of the ad increases pageviews of it, but it draws traffic away from something more shareable, reducing traffic to the site as a whole. Is there a way to make all of the downstream causal effects obvious? Is there a way to make causal assumptions explicit, so their merits can be discussed, and our model of the world can improve?  ... "  

Learning by Analogy

Particularly interesting suggestion of how people are 'intelligent'.

Making Computers Reason and Learn by Analogy

Structure-mapping engine enables computers to reason and learn like humans, including solving moral dilemmas    by Amanda Morris
 
Northwestern Engineering’s Ken Forbus is closing the gap between humans and machines.

Using cognitive science theories, Forbus and his collaborators have developed a model that could give computers the ability to reason more like humans and even make moral decisions. Called the structure-mapping engine (SME), the new model is capable of analogical problem solving, including capturing the way humans spontaneously use analogies between situations to solve moral dilemmas.

“In terms of thinking like humans, analogies are where it’s at,” said Forbus, Walter P. Murphy Professor of Electrical Engineering and Computer Science in Northwestern’s McCormick School of Engineering. “Humans use relational statements fluidly to describe things, solve problems, indicate causality, and weigh moral dilemmas.” ... ' 

See more on Structure Mapping Engines (SME).

Think More about the Causality

Nicely done, and for the method I often give for driving value in analytics, bring your business reasoning closer to the analytics.   Its the business that is of most importance.

In Yanir Seroussi's blog: 
Why you should stop worrying about deep learning and deepen your understanding of causality

Everywhere you go these days, you hear about deep learning’s impressive advancements. New deep learning libraries, tools, and products get announced on a regular basis, making the average data scientist feel like they’re missing out if they don’t hop on the deep learning bandwagon. However, as Kamil Bartocha put it in his post The Inconvenient Truth About Data Science, 95% of tasks do not require deep learning. This is obviously a made up number, but it’s probably an accurate representation of the everyday reality of many data scientists. This post discusses an often-overlooked area of study that is of much higher relevance to most data scientists than deep learning: causality. .... "  

See also this recent book. Why: A Guide to Finding and Using Causes by Samantha Kleinberg.