Topic 6C - Decision Trees

Learning Objectives

By the end of this topic, you should be able to:

• Explain what a decision tree is, how it classifies data, and how it is built from labeled training examples.
• Explain what overfitting is and why it is a problem in machine learning.

Learning Activities

To help you meet the learning objectives, we have prepared three readings. Please complete them in order.

Readings

• Reading 1 - What Is a Decision Tree? — the structure of a decision tree, how to read and trace one, and a fully worked classification example
• Reading 2 - Building a Tree from Data — where decision trees come from, what makes a good split, and the overfitting problem
• Reading 3 - Decision Trees in the World — where decision trees appear in real systems, their strengths and limits, and the bridge to supervised learning

These readings intentionally build on each other, so please complete them in order.

Checking for Understanding

Review the Learning Objectives at the top of this page. The questions below will help you check your understanding before moving on to Topic 6D.

Reading and Tracing Trees

1. Consider a simple decision tree for recommending whether a student should attend after-school tutoring:
   • Root split: Is the student's current grade below 70%?
   • If Yes → Is the student currently missing more than two assignments?
     • If Yes → Recommend tutoring
     • If No → Schedule a check-in conversation
   • If No → No action needed
   Trace this tree for a student with a 65% grade and all assignments submitted. What recommendation does the tree produce? Does it seem reasonable?
2. What is the difference between an internal node and a leaf node in a decision tree? What kind of information does each one represent?
3. A decision tree produces a classification for every input, even inputs that look nothing like anything in its training data. Is that a strength or a weakness? Explain your reasoning.

Building Trees and Overfitting

4. A loan officer is building a decision tree to classify loan applications as approve or deny. The available attributes are: credit score, annual income, existing debt, employment status, loan amount, and loan purpose (car, home, personal). Which attribute would you expect to produce the best first split, and why? Which attribute would you expect to produce the worst first split?
5. A teacher builds a decision tree from three years of student data to predict which students will struggle on the final exam. The tree perfectly classifies every student in those three years. When applied to this year's students, it performs no better than a coin flip. What is the most likely explanation? What does this tell us about what "learning" means for an AI system?
6. In your own words, explain why a simpler decision tree often generalizes better to new data than a very complex one. How is this similar to the way a good teacher writes a test — testing understanding rather than memorized facts?

Real-World Connections

7. Credit scoring, medical diagnosis support, spam filtering, and student early warning systems all use decision trees or related structures. Pick one of these and describe what it is classifying, what attributes it likely uses to split, and what the leaf node outcomes represent.
8. Decision trees are an example of supervised learning. In one or two sentences, explain what "supervised" means in this context — what role do the labeled training examples play in producing the tree?

It is completely fine to revisit the readings as you work through these questions.

Extend Your Learning