Thinking Causality in Science and Statistics

I have been looking back to understand how AI has changed since the 90s, when we worked with rule based expert systems.  One book that addresses some of the changes is Judea Pearl's:  Causality: Models, Reasoning and Inference.  Now over a decade old, it contains some interesting gems. Dealing with the mixing of knowledge in diagrams and equations, and developing approaches that have evolved to now commonly used Bayesian Networks.  More on his site.

There is also a copy of a lecture that Pearl gave at the time:  The Art and Science of Cause and Effect, originally an epilogue in the book.   Now available free at the link. Deals with the interesting concept of Causality, which is remarkably complex.  The idea is essential in working engineered systems, avoided in the physical sciences, and most always warned against in statistics.  The article examines why, and poses some remedies.  I disagree with some of his early historical views, causation was not discovered at the time of Galileo, but the lecture is still an excellent read.

Consider also how Big Data methods have backed off the need for strict causation requirements.