Using Data from Tracking Studies

Friend Kaiser Fung looks at some of the tracking data is being used in studies.  Instructive about how data is being gathered and used.   Extensive piece at the link.

The first major study using Covid tracking app data stumbles out of the gate

The tracking app studies are on rush order, just like all sorts of Covid-related preprints that have come under scrutiny. I learned about this collaboration between King’s College (UK) and an app developer Zoe Global through a news article proclaiming that 13 percent of the UK have already been infected with the novel coronavirus. This is the catch of the moment, being able to declare that you know what proportion of the population has already been infected.

I discussed the Stanford study the other day, which has received robust criticism from the statistical community (e.g. Gelman). This Covid Symptom Tracker study is much less convincing as a way to measure population prevalence. Its potential value is for rationing test kits to those most likely to test positive but notice that the objective of targeting the most afflicted conflicts with the objective of measuring prevalence, which is a feature of the general population.  .... "

Risk:Challenging Probability

Kaiser Fung on the Book  ... Risk,    by John Adams  Which has some interesting thoughts about probabilities describing risk.  He begins:

I have been reading the excellent book by John Adams titled Risk (link). This is a geographer's treatment of a subject that is a staple of mathematics, particularly probability math. A mathematical treatment creates objects called probability distributions, which are then taken as complete representations of risk. Adams challenges that construct, bringing a social scientist's sensitivity to the table. In particular, he points out how the mathematics of risk is undermined by measurement issues (i.e. data issues) and statistical issues. He is not invalidating math, just pointing out large cracks that are often ignored.

I will provide a more comprehensive review of the book eventually. I'm very excited by Chapter 5, titled "Measuring Risk", and specifically the example of "traffic black spots". This example is very instructive for anyone who is interested in the practical implications and interpretation of risk measures.

The post got long so I have split it into two parts. The second part will be posted on Monday, and it concerns a delicious bit of analysis related to traffic black spots.   .... " 

Kaiser Fung on the Limitations of AI

Colleague Kaiser Fung  on on hype and limitations of AI today.   Some useful thoughts.  In his always interesting Junkcharts blog.  I too have lived through several waves of AI hype, seeing it from an enterprise perspective.   Points to a number of studies.  In part this does depend on your definition of AI.

Primer on the limitation of current AI

Today, I turn attention to another Technology Review article in the same special AI issue, the important article by Rodney Brooks on the limitation of the current AI systems, driven by the "deep learning" "revolution." Brooks was the head of MIT's CSAIL lab, and an expert in AI who has lived through several waves of AI hype.

Brooks frames his content differently, titling the piece "The Seven Deadly Sins of AI Predictions." He opens with the breathless forecasts published by various media outlets, predicting that AI (computers) would wipe out the job market for human beings in a broad range of industries, in some cases, in a matter of 10 or 20 years. He then describes seven fallacies that have led these hype-meisters astray. Read the whole article here. Some of these dire forecasts: Oxford/Yale, McKinsey. [PS. I do not understand why those technology leaders who claim to believe in these forecasts do not immediately stop their AI programs for the public good.]

Brooks calls these predictions "ludicrous." He points out that there have been zero realistic demonstrations of robots that can take over grounds and maintenance work even though the forecasters claim that 90% of such jobs would disappear in 10 to 20 years. For the rest of us, who are not knee-deep in AI research, it's simple to validate Brooks's viewpoint: just make a call to your favorite service provider (your bank, your credit card provider, your health insurer, for example); subject yourself to the AI "chatbot" for even five minutes. 

In debunking the hype, Brooks outlines several key limitations of the current AI systems, built on the deep learning revolution (discussed in the prior post).   .... " 

( See also his previous piece on this, pointed to in the above) 

(Simple) Data Science in the Press

My colleague Kaiser Fung, in his Junkcharts Blog,  on the use of dubious fractured statistics in the press.  The NYT notably has created a 'data science' team.  Considering the notion of the median vs the average in effectively describing data in the press.   As always, instructive.

Columbia Master of Science in Applied Analytics

Via Kaiser Fung
" ... We have a bunch of classes in the "leadership, management, and
communications" area, listed here:
http://ce.columbia.edu/applied-analytics/master-of-science-in-applied-analytics/courses  ... " 

A conversation continues.  More to follow.

Fung Group Lab for Shopper Behavior

This was mentioned to me in a conversation recently, now the first reports from people who have visited.   Quite a bit of leading edge tech, including 3D printing and VR is also included.   Note especially the work with omnichannel interaction, which we rarely address in lab settings.  For More on related work in China, see the Hualian tag below.

In Retailwire:
" ... The shopping lab is devoted to analyzing omnichannel consumer behavior and is operated by the giant Fung Group (a mega-conglomerate with a strong presence in retailing and operators of Toys "R" Us, Circle K, and Gieves and Hawkes, to name a few, in Asia.), in partnership with IBM (a RetailWire sponsor) and brand activation firm Pico. The "lab" is actually 250,000 square feet of space at Li Fung Plaza, where select members can shop and businesses can observe how consumers interact with new tech, products and environments. There is a plethora of stores and customer behavior is analyzed by beacons and other store-monitoring devices.  ... " 

Columbia University MS in Applied Analytics

Kaiser Fung reports on the Columbia University Master of Science Program in Applied Analytics being established.   Their new site. 

Junk Charts: Data Visualizations Reviewed

Recently connected with Kaiser Fung,   Who writes the famous Junk Charts Blog that I have often mentioned here.  I had read about bad visualizing years ago in books like:  How to Lie with Statistics, But never saw a source that systematically looked at criticizing and improving current data visualization.  Kaiser brought that home to me.  Taught me many things.  Thanks Kaiser!

Here is his blog intro:  (Go to the link for details)

Junk Charts is made by Kaiser Fung, the Web’s first data visualization critic. I discuss what makes graphics work, and how to make them better. Think chartjunk + junk art.

Here is my first post from nine years ago. These are the posts with the most (clicks): 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. There are some things I say over and over again: the Trifecta Checkup, the self-sufficiency test, bumps charts, small multiples, the power of aggregation. I carefully tag all posts with topics so clicking on map shows all posts about maps. A list of topics is on the right column. Here is a call for more dataviz criticism. ... 

Here is my other blog, about statistics and Big Data. Here is my twitter feed, aggregating both blogs. If you prefer RSS, there's one feed for Junk Charts, and one for Big Data Plainly Spoken.  .... " 

See also his recent book:  Numbersense, which I read this year.  Great stuff for anyone that deals with numbers.

Statistical Significance

Kaiser Fung takes a core value principle look at the meaning of statistical significance.  His statement is very true, but is it actually useful in practice, except as a general caution?   What does it tell me to do with the models I have derived?   Give me something  I can work with.

NumberSense Reviewed

I have been looking for books that explain the difference of what I call analytics, in my long time consulting practice and the newly hyped arena of Big Data.   I am always also on the look out for books that I can use with client groups to explain how analytics is powerful.    The explanation has to be positioned with little or no complex mathematics.  The examples have to be clear, and beg for their simple reapplication to new business domains.

NumberSense includes these characteristics.   Easy to understand, non technical examples.  In the social/marketing/economic/sports domains.  Clear positioning about how the problems should be staged.  Not much about how the problems are technically solved, but that is for the data technologists.  Not a how to book, but sets up the crucial cautions very clearly.

I particularly liked the analysis of Groupon data, which clearly defines where claims and analyses can be wrong in marketing.   A good marketing analysis example.

Fung appreciates the fact that while having more, or 'big' data is useful, but it is more important to get the data and its analysis right, especially as it relates to the decision problem being addressed.  Numbersense is paying attention to the origin and context of the data involved, and knowing enough about how the analysis will be applied to the real problem.  Misinterpretation is the worst mistake you can make.

In the final chapter Fung describes a  day in his life as a data scientist.  This was painfully reminiscent of some of my own enterprise experiences.  Its often more difficult getting the data right than solving the technical problem.

As a decision oriented person you don't need to know the technical methods, any more than you need database expertise to create reports.  This books aims at the business problem and solutions, with a strong numerical focus.  Usually with basic math.   The title of the book NumberSense, is that quality of understanding when an analysis is right or going wrong, and what to do about that.   The data in an analysis does not have to be BIG or even complex, just correctly addressed.   The book and more about it:

NumberSense: How to Use Big Data to Your Advantage    by Kaiser Fung .

See also his Numbers Rule Your World blog site for day to day examples.

Examples covered: 

" ... How does the college ranking system really work?
Can an obesity measure solve America's biggest healthcare crisis?
Should you trust current unemployment data issued by the government?
How do you improve your fantasy sports team?
Should you worry about businesses that track your data?

Don't take for granted statements made in the media, by our leaders, or even by your best friend. We're on information overload today, and there's a lot of bad information out there.
Numbersense gives you the insight into how Big Data interpretation works--and how it too often doesn't work. You won't come away with the skills of a professional statistician. But you will have a keen understanding of the data traps even the best statisticians can fall into, and you'll trust the mental alarm that goes off in your head when something just doesn't seem to add up.... "

Short Non Mathematical Statistics Course

Nice idea, Kaiser Fung from Junkcharts.   I have often been in the need of having concise statistics training for managers and executives.  "How to do statistics without really doing statistics?"  .... Not a math course.  Have not looked at this, but will.

This also introduced me to the training system and construct called 3 Nights and Done, which I like the general premise of.  " ... It's free. It's non-profit. It won't sell your student activity records. It makes learning fun for people of all ages. You watch 1 hour of educational material each night, and in 3 nights' time it's done. Simple as that. You will get a sense of accomplishment instantly over say a weekend. ... " 

Averages

Good piece on averages, pulled from the new book: Numbers Rule Your World, by the author of that book Kaiser Fung,  which I did a pre review of here.  This is a good place to start in learning about practical uses of statistics.

Numbers Rule Your World

Numbers Rule Your World: The Hidden Influence of Probabilities and Statistics on Everything You Do by Kaiser Fung

In the midst of reading this book which I have had on the stack for some time.   A largely non-technical work that I would recommend to those that do not have a statistics background but still have to defend and describe analytical works to others.   Or to those with a statistics background who need to have good explanatory paragraphs to share.  A case study oriented book.  The studies I have seen so far are good, though I would have liked some more details of the technical solutions, perhaps in an appendix.  The author is a statistician who works in the advertising and consumer behavior space.  Recommended.

Numbers Rule Your World

A good review of the new book:  Numbers Rule Your World: the hidden influence of probability and statistics on everything you do ,   by Junk Charts blogger Kaiser Fung.  The book, which I have yet to see, is an overview of statistics for the non mathematician.  On my list of books to review in more detail.  I think it is very important to have books of this type for the management decision maker.  Especially important as more statistical analyses appear on dashboards.   On displays of these types I often got the question:  Explain to me again how much I can trust this analysis?

Junk Chart Book

The author of the always interesting Junk Chart Blog, Kaiser Fung, now has a book out. Described in the blog and now available: Numbers Rule Your World: The hidden influence of probability and statistics on everything you do. Plan to read and review.