Mini Tutorial on Probability

A nicely done mini tutorial, very clearly done,  essential stuff.  The intro to the book and Bayesian also very useful.

How to Develop an Intuition for Joint, Marginal, and Conditional Probability  by Jason Brownlee on September 30, 2019 in Probability

Probability for a single random variable is straight forward, although it can become complicated when considering two or more variables.

With just two variables, we may be interested in the probability of two simultaneous events, called joint probability: the probability of one event given the occurrence of another event called the conditional probability, or just the probability of an event regardless of other variables, called the marginal probability.

These types of probability are easy to define but the intuition behind their meaning can take some time to sink in, requiring some worked examples that can be tinkered with.

In this tutorial, you will discover the intuitions behind calculating the joint, marginal, and conditional probability.

After completing this tutorial, you will know:

How to calculate joint, marginal, and conditional probability for independent random variables.
How to collect observations from joint random variables and construct a joint probability table.
How to calculate joint, marginal, and conditional probability from a joint probability table.
Discover bayes opimization, naive bayes, maximum likelihood, distributions, cross entropy, and much more in my new book, with 28 step-by-step tutorials and full Python source code.

Let’s get started: ....

Certainty is Unusual

Jason Brownlee does his usual good job of explaining important concepts.   Here largely non technical.  And this is perhaps the most important.  Often the hardest to explain to decision makers, despite the fact that they deal with the problem every day.  Risk must always be considered.   Heartily recommend you subscribe.  See the 'Brownlee' tag below for other tutorials from Jason I have mentioned.

What Is Probability?  by Jason Brownlee  

Uncertainty involves making decisions with incomplete information, and this is the way we generally operate in the world.

Handling uncertainty is typically described using everyday words like chance, luck, and risk.

Probability is a field of mathematics that gives us the language and tools to quantify the uncertainty of events and reason in a principled manner.

In this post, you will discover a gentle introduction to probability.

After reading this post, you will know:

Certainty is unusual and the world is messy, requiring operating under uncertainty.
Probability quantifies the likelihood or belief that an event will occur.
Probability theory is the mathematics of uncertainty.

Let’s get started. ... 

Odds vs Probability

Good thought, since I am from an engineering background, I hear probability rather than odds.

Are you mixing up odds with probability?  in TowardDataScience

Odds and probability are different, and too many people make decisions without knowing that.
Keith McNulty

In day-to-day life people use the words ‘odds’ and ‘probability’ interchangeably. They are both terms that imply an estimate of chance. I also see these terms used interchangeably in the workplace. People can say that the ‘odds are twice as high’, and they can understand that to mean ‘the probability is double’. Well, that’s wrong!

Odds and probability are related concepts but very different in scale and meaning. When mixed up in the wrong contexts this can lead to mistaken estimates of chance, which can then lead to erroneous decision making.

In this article, I want to illustrate what those differences are and how, in confusing the two, you can really affect analysis and research.

What is the difference between probability and odds?  .... "

The Odds of Strategy

Have not read this book, but the premise is interesting.  But is strategy in context usually too broad to pin good odds on?

Strategy to beat the odds  By Chris Bradley, Martin Hirt, and Sven Smit  ...  in McKinsey Quarterly 

If you internalize the real odds of strategy, you can tame its social side and make big moves.
Several times a year, top management teams enter the strategy room with lofty goals and the best of intentions: they hope to assess their situation and prospects honestly, and mount a decisive, coordinated response toward a common ambition. ...

In our book, Strategy Beyond the Hockey Stick: People, Probabilities, and Big Moves to Beat the Odds (Wiley, February 2018), we set out to help companies unlock the big moves needed to beat the odds.  ... "

More Bayesian Networks

Video, another example of using Bayesian networks.

Dr. Alta de Waal's presentation at the 5th Annual BayesiaLab Conference in Paris.

Spatially Discrete Probability Maps for Anti-Poaching Efforts

Via Stefan Conrady
Managing Partner at Bayesia USA & Singapore: Bayesian Networks  ...

A Bayesian Learning Framework

    I particularly like the premise here ... that the approach starts with causal decision descriptions of the real world.  With the potential of building from real world process.   Often use 'influence diagrams' to do this.  Or existing process models.  Sounds like there could be a means to iterate between real world and model.  And converge?

In Technology Review:
" ... The researchers used a technique they call the Bayesian program learning framework, or BPL. Essentially, the software generates a unique program for every character using strokes of an imaginary pen. A probabilistic programming technique is then used to match a program to a particular character, or to generate a new program for an unfamiliar one. The software is not mimicking the way children acquire the ability to read and write but, rather, the way adults, who already know how, learn to recognize and re-create new characters.

“The key thing about probabilistic programming—and rather different from the way most of the deep-learning stuff is working—is that it starts with a program that describes the causal processes in the world,” says Tenenbaum. “What we’re trying to learn is not a signature of features, or a pattern of features. We’re trying to learn a program that generates those characters.”  .... ' 

The Banburismus Process and Joan Clarke

After seeing and reviewing the movie about Alan Turing movie: The Imitation Game,  I took my research obsession to looking further at Joan Elisabeth Clarke,  Turings sometime fiance, and colleague at Bletchley Park during WWII.  She was also an award winning Cambridge mathematician. Portrayed in the movie by actress Keira Knightley.

In her WP article it was mentioned that she was a cryptanalyst and a practitioner of what was called Banburismus,  a method used to simplify code breaking. Expanded by Turing and possibly improved by Clarke and other members of the team. Some of this work is still considered confidential.

There is a lengthy article about the Banburismus  process in the WP.    A fascinating look at how a polyalphabetic substitution cypher based on the wheels of the Enigma machine could be simplified with a hand and visualizing algorithm. The article even gives several examples.  Leads to an appreciation of how difficult this method was without programmable computing.  Also an early look at text analytics.

Further, it is mentioned that this was a development of the 'clock method', developed by Polish cryptanalyst Jerzy Rozycki.    Pointing to the key work done, barely mentioned in the movie, by the Polish group.

Visualizing a Conditional Probability

This is a visual simulation of conditional probability.   The description at the link is quite technical and mathematical, but the visual simulation there provides a good example of what is happening.   I like these kinds of visuals, useful for explanation with non technical decision makers.   -  via Ramon Martinez- Advisor in Health Metrics, Noncommunicable Diseases and Mental Health, Pan American Health Organization (PAHO).  

I see that the creator of this: Setosa does a number of web and data visualizations and animated explanations, more examples here.

Numbers Rule Your World

A good review of the new book:  Numbers Rule Your World: the hidden influence of probability and statistics on everything you do ,   by Junk Charts blogger Kaiser Fung.  The book, which I have yet to see, is an overview of statistics for the non mathematician.  On my list of books to review in more detail.  I think it is very important to have books of this type for the management decision maker.  Especially important as more statistical analyses appear on dashboards.   On displays of these types I often got the question:  Explain to me again how much I can trust this analysis?