A Different Kind of Learning Problem
In supervised learning, every training example comes with a label — the correct answer the system is trying to learn to predict. The label is the teacher. Without it, there is no signal to learn from.
Unsupervised learning removes the labels entirely. The system receives data and nothing else. No correct answers. No categories defined in advance. The task is not to predict a known outcome but to discover what structure exists in the data — groupings, patterns, relationships, and regularities that were not specified by any human.
This sounds like a harder problem, and in some ways it is. But it is also a more powerful one, because unlabeled data is everywhere. Every photograph ever taken, every sentence ever written, every transaction ever recorded exists as unlabeled data. Labeled data — data that a human has taken the time to annotate with correct answers — is rare, expensive, and always in short supply. Unsupervised learning is how AI systems can learn from the vast ocean of data that has no labels attached.
Clustering: The Core Unsupervised Task
The most intuitive unsupervised learning technique is clustering: grouping data points into categories based on similarity, without any predefined category labels. The algorithm finds the groups; the human decides what they mean.
You encountered clustering in Week 5 as one of the six data mining techniques. Recall that cluster analysis discovers groups that were not defined in advance — as opposed to class description, which characterizes groups that already exist. Topic 7b provides the algorithmic foundation beneath that concept.
The most widely used clustering algorithm is K-means. It is simple enough to trace by hand, robust enough to be used in production systems, and illustrative of the core challenge of unsupervised learning: finding meaningful groupings in data without anyone telling you what the groups should be.
How K-Means Works
K-means requires one piece of human input before it begins: the number of clusters, K. You must decide in advance how many groups you are looking for. The algorithm then finds the K groups that best partition the data.
The algorithm proceeds through a simple cycle:
- Initialize: Place K centroids randomly in the data space. A centroid is a point that will become the center of one cluster.
- Assign: Assign each data point to the nearest centroid. Every point belongs to exactly one cluster.
- Update: Recalculate each centroid as the average position of all points currently assigned to it.
- Repeat: Go back to step 2. Continue until the assignments stop changing — the algorithm has converged.
The result is K clusters, each defined by its centroid, with every data point assigned to the cluster whose centroid is closest.
Worked Example: Student Engagement Clustering
A school counselor has data on eight students, each described by two attributes: their average daily attendance rate (as a percentage) and their assignment completion rate (also as a percentage). She wants to use K-means with K=2 to find two natural groups in the data.
| Student | Attendance % | Completion % |
|---|---|---|
| A | 95 | 92 |
| B | 88 | 85 |
| C | 91 | 90 |
| D | 72 | 55 |
| E | 65 | 48 |
| F | 93 | 88 |
| G | 70 | 60 |
| H | 68 | 52 |
Pass 1 — Initialize
The algorithm randomly places two centroids. Suppose it starts with Centroid 1 at (95, 92) — coinciding with Student A — and Centroid 2 at (65, 48) — coinciding with Student E.
Pass 1 — Assign
Each student is assigned to the nearest centroid. We measure distance using straight-line (Euclidean) distance between the two points in attendance-completion space.
| Student | Distance to C1 (95,92) | Distance to C2 (65,48) | Assigned to |
|---|---|---|---|
| A (95,92) | 0 | 53.8 | C1 |
| B (88,85) | 9.9 | 46.5 | C1 |
| C (91,90) | 4.5 | 49.2 | C1 |
| D (72,55) | 50.6 | 10.3 | C2 |
| E (65,48) | 53.8 | 0 | C2 |
| F (93,88) | 4.5 | 50.1 | C1 |
| G (70,60) | 44.7 | 12.8 | C2 |
| H (68,52) | 52.2 | 4.5 | C2 |
Pass 1 — Update
Recalculate centroids as the average position of assigned points:
- Cluster 1 (A, B, C, F): Average attendance = (95+88+91+93)/4 = 91.75, Average completion = (92+85+90+88)/4 = 88.75. New C1 = (91.75, 88.75)
- Cluster 2 (D, E, G, H): Average attendance = (72+65+70+68)/4 = 68.75, Average completion = (55+48+60+52)/4 = 53.75. New C2 = (68.75, 53.75)
Pass 2 — Assign and Update
With the updated centroids, every student is reassigned. In this case, the distances shift slightly but the same four students remain in each cluster. The assignments have not changed, so the algorithm has converged.
What K-Means Cannot Do
K-means is powerful and fast, but it has genuine limits worth understanding:
- K must be chosen in advance. The algorithm cannot determine how many clusters are appropriate. If you specify K=3 on data that has two natural groups, the algorithm will produce three clusters anyway — and some of them will be artificial. Choosing K requires domain knowledge or additional techniques for estimating the right number of groups.
- Results depend on initialization. Different random starting centroids can produce different final clusters. Running K-means multiple times with different starting points and taking the best result is standard practice.
- Clusters are always spherical. K-means assumes clusters are roughly circular in shape. Data with elongated, curved, or irregularly shaped natural groupings will be poorly captured by K-means, regardless of K.
- Labels are still a human job. K-means finds groups; it cannot explain what those groups mean. Two different analysts looking at the same clusters might name them very differently depending on their knowledge and perspective.
In Reading 2, we look at a different unsupervised approach: rather than finding groups, finding a simpler representation of the data that reveals its hidden structure and makes it accessible for visualization and further analysis.