Small Data Also Crucial, With Tested Transfer Assumption

We tried related ideas but found that the transfer assumption rarely held well enough to be useful.  Useful to keep in mind to potentially simplify models, with care..

'Small Data' Is Also Crucial for Machine Learning

By Scientific American, October 19, 2021

The existence of techniques such as transfer learning does not seem to have reached the awareness of policy makers and business leaders making decisions about AI funding and adoption..... 

Some of the most prominent artificial intelligence (AI) breakthroughs in the past decade have relied on enormous data sets. But AI is not only about large data sets; research in "small data" approaches has grown extensively over the past decade—with so-called transfer learning as an especially promising example.

Small-data approaches such as transfer learning offer numerous advantages over more data-intensive methods. Enabling the use of AI with less data can bolster progress in areas where little or no data exists, such as in forecasting natural hazards that occur relatively rarely or in predicting the risk of disease for a population set that does not have digital health records.

From Scientific American

View Full Article

Small Data, Short Term Global Risks, GRR

 Nicely put, perhaps on every risk analysis we should list these calculate how they might be of threat to our contexts.   At the link below some good charts and analysis, plus links to much deeper analyses.

January’s most engrossing Small Data

The Global Risks Report   By David Aldous

As a mathematician of the old school, I don’t seek to engage Big Data. Instead, this post is about Small Data on a Big Subject. Each January since 2006 the annual Global Risks Report (GRR) has been published, as background material for the annual World Economic Forum (see Footnote 1 below). The reports are lengthy documents, freely available here  analyzing risks in the sense of events that would have substantial effect on the world economy over the next few years (“medium term”). The reports provide a consensus view derived from a large panel of experts. For my purpose here, the centerpiece is a graphic showing the perceived likelihood and economic effect of each of 36 risks (see Footnote 2). Look at this graphic below from January 2020 (or click this link).   .... ' 

Become a Marketing Detective: Induction to Deduction

More takes on the idea of 'Small Data' proposed by my former collaborator Martin Lindstrom in his recent book: Small Data: Tiny Clues that Uncover Huge Trends ... "    Another way to look at this is as an 'inductive' approach, where you are selectively looking for especially valuable patterns in data.  That method can be used for data, large or small.

In Big Data,  statistics is the usual tool of choice to make sense of the complexity.   With less data, even small data, you need the insight of a detective to find patterns that are useful, or can be combined with your internal insight rules of thumb to get to results.  That's deduction.  That's being a marketing detective.   The article linked to below takes such a Sherlock Holmesian approach to some examples for innovative marketing.

Why can this be more interesting than the world of 'Big Data'?  Because it can take the leverage closer to the 'business processes' we use every day.  More sense and less magic is easier to implement and appreciate.

See:   strategy+business: Corporate Strategies and News Articles on Global Business, Management, Competition and Marketing  ... A Not-So-Elementary Exploration of Brand Insight ...  " 

Martin Lindstrom Podcast on Small Data

Martin Lindstrom discusses his new book about 'small data'

Podcast: 
Small Data coverMartin Lindstrom has spent time with 2,000 families in more than 77 countries to get clues to how they live — resulting in the acquisition of what he likes to call Small Data. In his new book, Small Data: The Tiny Clues That Uncover Huge Trends, he argues that the Small Data explains the why behind what Big Data reveals. Knowledge@Wharton recently spoke with Lindstrom on the Knowledge@Wharton show on Wharton Business Radio on SiriusXM channel 111.

An edited transcript of the conversation follows. ... "

Now how can such data be connected with structured and unstructured data?  Is this about the story,  or is there a deeper connection.

Big Data Isn't Enough

Colleague Martin Lindstrom's recent book:  Small Data: The Tiny Clues that Uncover Huge Trends, was recently covered in a Fortune article.    " .... It can’t give you the whole picture .... Here are 5 reasons why Big Data no longer can stand alone, and why the future is likely to always include Small Data .... "   

The Master storyteller looks at the small indicators:  

" ... Hired by the world's leading brands to find out what makes their customers tick, Martin Lindstrom spends 300 nights a year in strangers’ homes, carefully observing every detail in order to uncover their hidden desires, and, ultimately, the clues to a multi-million dollar product.

Lindstrom connects the dots in this globetrotting narrative that will enthrall enterprising marketers, as well as anyone with a curiosity about the endless variations of human behavior.  ... " 

Is Small Data more Business Process Context?

Making my way through the book below.  Martin Lindstrom is a great storyteller, starting with his well known work with LEGO.      Before I finish the book, my initial impression is that there is real value here.  

It can be seen as making sure you include as much real business context in your problem as possible. In my usual world this means including it in your data science, but it can also mean starting with business process analysis, and adding the deep analysis.

Is that Small Data?   Simple insight? As long as it produces value, that is good.  And getting the business decision makers included, and valued, as soon as possible is a way to get your method understood and delivered.   Will have more on this as I proceed.    Worth a read. Thoughts?

 ...  Former colleague Martin Lindstrom has a new book out: Small Data: The Tiny Clues That Uncover Huge Trends .  .....   Take a look at their pre publicity.

Martin Lindstrom: Small Data

 I see that former colleague Martin Lindstrom has a new book forthcoming: Small Data: The Tiny Clues That Uncover Huge Trends .  Have not seen it yet, but I like the premise.  Take a look at their pre publicity.