Exploiting Multi-Categorical Features

Had not heard of this in particular, have passed it on.

Exploiting Multi-Categorical Features Using Deep Interest
By Marina Gandlin | Data Science
Tags: algorithms, big data, data, data model, data science, deep learning, machine learning, neural networks

At Taboola, our goal is to predict whether users will click on the ads we present to them. Our models use all kinds of features, yet the most interesting ones tend to be related to the users’ history. Understanding how to use these features well can have a huge impact on the model’s personalization capabilities, due to the user-specific knowledge they hold.  ..... ' 

Feature Visualization

In the Google Blog, they provide an overview and pointer to deeper information.  An image rich and informative article.

Feature Visualization
Posted by Christopher Olah, Research Scientist, Google Brain Team and Alex Mordvintsev, Research Scientist, Google Research

Have you ever wondered what goes on inside neural networks? Feature visualization is a powerful tool for digging into neural networks and seeing how they work.

Our new article, published in Distill, does a deep exploration of feature visualization, introducing a few new tricks along the way!

Building on our work in DeepDream, and lots of work by others since, we are able to visualize what every neuron a strong vision model (GoogLeNet [1]) detects. Over the course of multiple layers, it gradually builds up abstractions: first it detects edges, then it uses those edges to detect textures, the textures to detect patterns, and the patterns to detect parts of objects….   " 

" .... There is a growing sense that neural networks need to be interpretable to humans. The field of neural network interpretability has formed in response to these concerns. As it matures, two major threads of research have begun to coalesce: feature visualization and attribution. ... " 

IOT Failure Prediction

Attended this webinar of interest, with a largely nontechnical approach, other parts will follow: " Industrial IoT and Failure Prediction on event signals | Part 1 " featuring Adurthi Ashwin Swarup, Senior Data Scientist, DataRPM.

Here's the replay link.

More on Feature Selection

A favorite topic of mine, and I believe the most important introductory aspect of analytic modeling.
Here in DSC.   It is very important to get feature selection right.  Both in what you include, and what you leave out.    In particular this article does a very good job in showing how the selection of features is influenced by your goals in modeling, and how your  models can be made more predictive by engineering your variables.  This links model goals and business needs, always a good idea.


In DSC:  " ... Feature selection is one of the core topics in machine learning. In statistical science, it is called variable reduction or selection.  ... Here, we mention an article published by Isabelle Guyon and Andre Elisseeff in Journal of Machine Learning Research. While published in 2003, it is still one of the best ML papers on feature selection. 

Abundance Theory Applied to Analytics

Abundance Theory: Had never heard it called this, and was always as a general mindset rather than a working strategy.   Analytics usually works with constraints, can algorithms take us beyond this? Can we 'feature engineer' ourselves beyond constraints?

" ..... Scarcity theory, a term coined by Stephen Covey, suggests that everything in life has its limit. Whether that thing is a spot on the team roster, a scholarship, a job, customers, funding, promotions or something else, we need to hoard as much as possible for ourselves because there is simply not enough to go around. This same theory also says that there are limited ways to achieve success, and that anyone who wishes to make it must follow the same path and prescription that others have done previously.

In contrast, this coach, through her word and deed, demonstrated to me a living illustration of what Covey labeled abundance theory, or AT. Abundance theory is a mindset that looks at each glass as half full (at least) and sees the world as offering endless opportunity. ... " 

Feature Engineering Tips

So what is feature engineering?  Fundamentally it is the most important part of data science.  Or of any analytic method, from the simplest to the most complex.  Features are the data variables you include in your model.  Or often more importantly, the data you leave out of your model.  Or how your data can be combined or 're-engineered' to make it most useful.  Often you do not know what data features you have, or can measure, or even exist at all.   So its a good idea to thoughtfully understand which are important.  DSC on engineering them, with link to a white paper.    by Patti Tillotson