Decision Trees in Machine Learning: Two Types (+ Examples)

Trees are a common analogy in everyday life. Shaped by a combination of roots, trunks, branches, and leaves, trees often symbolise growth. In machine learning, a decision tree is an algorithm that can create classification and regression models. 

The decision tree is so named because it starts at the root, like an upside-down tree, and branches off to demonstrate various outcomes. Because machine learning is based on solving problems, decision trees help us visualise these models and adjust how we train them.

What is a decision tree? 

A decision tree is a supervised learning algorithm for classification and regression modelling. Regression is a method used for predictive modelling, so these trees are used to classify data or predict what will come next. 

Decision trees look like flowcharts. They start at the root node with a specific data question and lead to branches that hold potential answers. The branches then lead to decision (internal) nodes, which ask more questions and lead to more outcomes. This continues until the data reaches a terminal (or 'leaf') node and ends.

In machine learning, there are four main methods of training algorithms: supervised, unsupervised, reinforcement learning, and semi-supervised learning. A decision tree helps us visualise how a supervised learning algorithm leads to specific outcomes.

For a more detailed look at decision trees, watch this video:

Introduction to supervised learning

If you want to deepen your knowledge of supervised learning, consider this course, Introduction to Supervised Learning: Regression and Classification, from DeepLearningAI and Stanford University. In 33 hours or less, you’ll get an introduction to modern machine learning, including supervised learning and algorithms such as decision trees, multiple linear regression, neural networks, and logistic regression.

Why is a decision tree important in machine learning?

Decision trees in machine learning provide an effective decision-making method because they lay out the problem and all the possible outcomes. It enables developers to analyse the potential consequences of a decision, and as an algorithm accesses more data, it can predict outcomes for future data. 

In this simple decision tree, the question of whether or not to go to the supermarket to buy toilet paper is analysed:

In machine learning, decision trees offer simplicity and a visual representation of the possibilities when formulating outcomes. Below, we will explain how the two types of decision trees work. 

Types of decision trees in machine learning

Decision trees in machine learning can be either classification trees or regression trees. Together, both algorithms fall into a category of “classification and regression trees” and are sometimes called CART. Their respective roles are to “classify” and to “predict.”

1. Classification trees

Classification trees determine whether an event happened or didn’t happen. Usually, this involves a 'yes' or 'no' outcome. 

We often use this type of decision-making in the real world. Here are a few examples to help contextualise how decision trees work for classification:

• Example 1: How to spend your free time after work

What you do after work in your free time can depend on the weather. If sunny, you can picnic with a friend, grab a drink with a colleague, or run errands. If it is raining, stay home and watch a movie instead. There is a clear outcome. That is classified as whether to “go out” or “stay in.”

• Example 2: Homeownership based on age and income

In a classification tree, the data set splits according to its variables. Two variables, age, and income, determine whether or not someone buys a house. If training data tells us that 70 percent of people over age 30 bought a house, then the data gets split there, with age becoming the first node in the tree. This split makes the data 80 percent 'pure.' The second node then addresses income.

To understand how decision trees work in machine learning, consider registering for these Guided Projects to apply your skills to real-world projects. You can complete them in two hours or less:

• Decision Tree and Random Forest Classification using Julia

• Predicting Salaries with Decision Trees

2. Regression trees

Regression trees, on the other hand, predict continuous values based on previous data or information sources. For example, they can predict the price of gasoline or whether a customer will purchase eggs (including which type of eggs and at which store).

This type of decision-making is more about programming algorithms to predict what is likely to happen, given previous behaviour or trends. 

• Example 1: Housing prices in West Bengal

Regression analysis could be used to predict the price of a house in West Bengal, plotted on a graph. The regression model can predict housing prices in the coming years using data points from previous years' prices. This relationship is a linear regression since housing prices will continue to rise. Machine learning helps us predict specific prices based on a series of variables that have been true in the past.

• Example 2: Bachelor’s degree graduates in 2025

A regression tree can help a university predict how many bachelor’s degree students will be in 2025. On a graph, one can plot the number of degree-holding students between 2010 and 2022. If the number of university graduates increases linearly each year, then regression analysis can be used to build an algorithm that predicts the number of students in 2025. 

To get started on how decision tree algorithms work in predictive machine learning models, look at these Guided Projects. Each project takes less than two hours, and they are based on real-world examples so you can elevate your skills:

• XG-Boost 101: Used Cars Price Prediction

• Decision Tree Classifier for Beginners in R

Classification and Regression Tree (CART) is a predictive algorithm used in machine learning that generates future predictions based on previous values. Decision trees are at the core of machine learning and serve as a basis for other machine learning algorithms, such as random forest, bagged decision trees, and boosted decision trees.

Decision tree terminology

These terms come up frequently in machine learning and are helpful to know as you embark on your machine-learning journey:

• Root node: The topmost node of a decision tree that represents the entire message or decision

• Decision (or internal) node: A node within a decision tree where the prior node branches into two or more variables

• Leaf (or terminal) node: The leaf node is also called the external node or terminal node, which means it has no child—it’s the last node in the decision tree and furthest from the root node

• Splitting: The process of dividing a node into two or more nodes; the part at which the decision branches off into variables

• Pruning: The opposite of splitting, the process of going through and reducing the tree to only the most important nodes or outcomes

Machine learning with Coursera

Start your machine learning journey with a top-rated Specialisation on Coursera, Supervised Machine Learning: Regression and Classification, offered by Stanford University and DeepLearning.AI. Taught by Andrew Ng, this course will provide the ultimate introduction to machine learning, where you will build machine learning models in Python using popular libraries NumPy and scikit-learn, and train supervised machine learning models for prediction (including decision trees!).