Benford's Law in the Universe

 We used Benford's law to detect potential fraud.  I have mentioned it here a number of times. Turns out its much more broadly observed.   Which is fairly obvious, math is also observed throughout the universe.   Fits my interests well.  Here is a good overview.  Worth understanding. 

Benford’s law and distances to stars  

Mysterious digit-law embedded in the universe?

By Jurjen de Jong in TowardsDataScience

One day Simon Newcomb (1835–1909) was looking at the pages of logarithmic tables when he saw that the first pages were more worn out than further pages. This simple observation and the way logarithmic tables are structured, meant that numbers starting with digit 1 were more common in nature than numbers starting with digit 2 and digit 2 more common than digit 3 and so on. This strange pattern in the first-digit frequency might sound counterintuitive at first, but it turned out to be true for many numerical datasets.

To make more clear what we mean with the first digit: this means that 1, 1213123, and 0.00153 all have digit 1 as the first digit, while 312, 0.3, and π share 3 as the first digit.

Newcomb published a paper about it in 1881, but his discovery wasn’t really picked up by the scientific community. So, when Frank Benford (1883–1948) rediscovered this phenomenon in 1938, he didn’t know about Newcomb’s discovery. Benford looked at the first digits of many different datasets, such as lengths of rivers, addresses, atomic weights, random newspaper numbers, and so on. Every time, he saw a similar pattern, which supported the idea that numbers could be sorted on their first digit. This digit-law is now called Benford’s law but could also have been called Newcomb’s law. .... '

Benford's Law and Data Science

Used it from the very beginning in enterprise data science.  well worth understanding,  especially for anomaly cases in finance or research fraud.   Even in finance we found relatively few people that had heard of it or how to use it.  Good, mostly non technical overview.

What is Benford’s Law and why is it important for data science?

By Tirthajyoti Sarkar
Sr. Principal Engineer | Ph.D. in EE (U. of Iilinois)| AI/ML certification (Stanford, MIT) | Data science author | Open-source contributor| AI in Simulations

We discuss a little-known gem for data analytics — Benford’s law, which tells us about expected distribution of significant digits in a diverse set of naturally occurring datasets and how this can be used for anomaly or fraud detection in scientific or technical publications.

Introduction
We all know about the Normal distribution and its ubiquity in all kind of natural phenomena or observations. But there is another law of numbers which does not get much attention but pops up everywhere — from nations’ population to stock market volumes to the domain of universal physical constants.

It is called “Benford’s Law”. In this article, we will discuss what it is, and why it is important for data science. 

What is Benford’s law?

Benford’s Law, also known as the Law of First Digits or the Phenomenon of Significant Digits, is the finding that the first digits (or numerals to be exact) of the numbers found in series of records of the most varied sources do not display a uniform distribution, but rather are arranged in such a way that the digit “1” is the most frequent, followed by “2”, “3”, and so in a successively decreasing manner down to “9”.  ... " 

Benford's Law and Greece in the EU

Intriguing application of the little known Benford's law.  A data science means for indicating falsified data.  I am a follower of its application in practice.  Could it have been used to prevent the Greece financial crisis?

Benford's Law for Twitter Data

Nicely done Tech Review piece on Benford's Law, a practical piece of pattern recognition knowledge every analyst should know.  It is used in fraud and risk analysis.  It finds strangeness in certain kinds of patterns.   Learned it early in my career.   Turns out it now has value even in the analysis of social media.  It's discovery is a fun case study by itself, which makes the point that you should aim to dive into the physical whenever given only abstract data:

" ... The counterintuitive distribution of digits in certain data sets turns out to be a powerful tool for detecting strange behavior on social networks.  It turns out it is even finding use in the analysis of social networks. 

Back in the 1880s, the American astronomer Simon Newcomb noticed something strange about the book of logarithmic tables in his library—the earlier pages were much more heavily thumbed than later ones implying that people looked up logarithms beginning with “1” much more often than “9.”
After some investigation, his concluded that in any list of data, numbers beginning with the digit “1” must be much more common than numbers beginning with other digits. He went on to formulate mathematical rationale behind this phenomenon, which later became known as Benford’s law, after the physicist Frank Benford who discovered it independently some 50 years later. ... " 

Following the quote is a look how it applies to social data.  Includes link to a technical paper.

Risk Analytics Day

The UC Center for Business Analytics will host Risk Analytics Day on February 11, 2015  8AM - 4:30PM at the Tangeman University Center (TUC) on the campus of the University of Cincinnati

Three speakers will present in the AM and the afternoon will be hands on risk analysis and simulation. Parking, breakfast. and lunch are included with registration.

 Risk Analytics Day Registration Here

Speakers
"The Flaw of Averages and How to Cure It"  Sam L Savage
"Benford’s Law as a Risk Analytics Tool"  Mark Nigirini
"Valid Models and Analysis of Risk in Complex Real-World Settings:  Simulation as a Method of Choice" David Kelton

Software Demonstrations:
Hands-On Simulation Modeling and Analysis with Arena 14.50.
SIPmath™ interactive Simulation in Excel: 10,000 Trials Before Your Finger Leaves the Key, Without Macros or Add-ins.

- See more here

@UCBusAnalytics