Building Decision Trees by Hand without Code

 Cool little piece that is instructional and even inspirational.   Math and process seen together to produce real value.

Random Forest and Decision Trees by hand — no coding

By Thomas Le Menestrel,      Computational Engineering student at Stanford | Passionate about Machine Learning and Data Science  in TowardsDataScience 

Introduction

In this article, we will discuss Decision Trees and Random Forest, two algorithms used in Machine Learning for classification and regression tasks.

I will show how to build a Decision Tree from scratch using a pen and paper and how to generalise this and build a Random Forest model.

Dataset

Let’s see how this work in practice with a simple dataset. .... ( much more )

Decision Trees and Random forests

This is fundamental stuff, that every practitioner should know well.  Its the simplest way that machines can be made to learn. And provides a good way to deliver real results.  Nice description... most is quite non-technical.  You should also know when this wont work, the data required, cautions to take .... but this is a great start

How do Decision Trees and Random Forests Work?

Part 1: Decision Trees

By Dick Brown  in TowardsDatascience

 Decision trees and random forests are two commonly used algorithms in predictive modeling. In this article, I’m going to discuss the process behind decision trees. I’m planning to follow this up with a second part that discusses random forests, and then compare the two.

First off: decision trees. A decision tree is named for the shape of the plot that comes out. The image below shows a decision tree for deciding what factors affected survival from the Titanic disaster.  ... " 

Tutorial: Random Forest for Time Series

 Am a long  time proponent of ensemble methods.  Here Jason Brownlee provides a nice tutorial on an often powerful method everyone should know.  As usual, well done, minimal tech.

Random Forest for Time Series Forecasting   by Jason Brownlee

by Jason Brownlee  in Time Series

Random Forest is a popular and effective ensemble machine learning algorithm.

It is widely used for classification and regression predictive modeling problems with structured (tabular) data sets, e.g. data as it looks in a spreadsheet or database table.

Random Forest can also be used for time series forecasting, although it requires that the time series dataset be transformed into a supervised learning problem first. It also requires the use of a specialized technique for evaluating the model called walk-forward validation, as evaluating the model using k-fold cross validation would result in optimistically biased results.

In this tutorial, you will discover how to develop a Random Forest model for time series forecasting.

After completing this tutorial, you will know:

Random Forest is an ensemble of decision trees algorithms that can be used for classification and regression predictive modeling.  Time series datasets can be transformed into supervised learning using a sliding-window representation.How to fit, evaluate, and make predictions with an Random Forest regression model for time series forecasting.

Let’s get started.   ....  

Random Forests Explained

A look at the method of Random Forests:  Random Forests Explained Intuitively.  Posted by Manish Kumar Barnwal.    If not quite intuitive, relatively non-technical.   With some related code to follow.    A kind of ensemble classification model.   Data hungry, as most methods are today.  In DSC, always an interesting and verbose source.  See the tag below for a lecture on the topic.

Lecture on Decision Trees

Video  " ... Random forests, aka decision forests, and ensemble methods. Slides available at: http://www.cs.ubc.ca/~nando/540-2013/... Course taught in 2013 at UBC by Nando de Freitas ... " 

Gathering in Random Forests

Random Forests. A notable ensemble-style method for analytic classification.  Particularly interesting to me because it uses more easily understood decision trees.     Some relationship to other clustering methods like K Means.  As it is available in R  Note that ensemble methods use multiple kinds (or parameters) of analytical models and compare their value on a given problem.  The approach is becoming increasingly used for tough problems.

Wal-Mart Trip Type Classification

From DSG:  Nice example of how data science can be used for a real world classification problem, with real data.  Including 1.2 million observations with 6 features.    Here using a Random Forest method for classification of retail trip types and comparing it to other methods.    Includes all the Python code from a Kaggle competition. Visualization of modeling progress.   Nicely done example. via Kirk Borne.

Ensemble Methods

Ensemble methods are means of using multiple approaches to solving an analytical problem.   Different tools can be better for different machine learning problems, or even different data.  Here is a thumbnail history and description.   Also the WP entry.  And a post on Random Forest methods.

Random Forests for Classification

A technical discussion of using Random Forest Algorithms for large database classification problems.    Have not used this myself yet, but I am dealing with classification systems.     " ... Both R and Python have robust packages to implement Random Forests.  ... "