Reading 2: The Six Techniques

A Family of Techniques

Data mining is not a single algorithm or a single tool — it is a family of techniques, each designed to discover a different kind of pattern. The right technique depends on what kind of structure you are looking for in your data. Are you trying to describe what a known group has in common? Discover whether natural groupings exist at all? Find unexpected connections between unrelated items? Identify records that do not fit the expected pattern?

This reading introduces six of the most important data mining techniques. For each one, we will define what it does, give a general example, and then give a school-relevant example — because these techniques are not abstract academic concepts. They are already being used in education, and your students will encounter them.

1. Class Description

Class description identifies the properties that characterize a known group of data items. You start with a defined group — a class — and the technique finds what the members of that group have in common.

General example: A car manufacturer wants to understand who buys their most fuel-efficient models. Class description applied to their customer database might reveal that buyers of these models tend to be between 35 and 55, live in urban areas, have above-average household incomes, and have previously owned at least one hybrid vehicle. That profile characterizes the class — it describes what the group looks like.

School example: A district wants to understand what characteristics students who are on track for on-time graduation share. Applied to several years of student records, class description might reveal that on-track students tend to have fewer than five absences in their freshman year, passed both algebra and English in ninth grade, and participated in at least one extracurricular activity. That profile can then inform early intervention efforts for students who do not fit it.

2. Class Discrimination

Class discrimination identifies properties that distinguish two groups from each other. Rather than characterizing one group, it finds what separates two groups.

General example: Using the same car dealership data, class discrimination would be used to find what distinguishes customers who buy new cars from those who buy used cars — not just describing each group separately, but finding the features that most reliably predict which category a customer falls into.

School example: A district applies class discrimination to distinguish students who responded positively to a reading intervention from those who did not. The technique might reveal that response to the intervention is strongly associated with the student's grade level at the time of intervention and whether they had received similar support in earlier grades — helping the district target future interventions more effectively.

3. Cluster Analysis

Cluster analysis discovers groups that were not predefined. Rather than characterizing or comparing known classes, cluster analysis asks: does this data naturally fall into distinct groupings, and if so, what are they? The groups — clusters — emerge from the data itself rather than from prior assumptions.

General example: A streaming music service analyzes its listeners without any predefined categories. Cluster analysis might reveal that listeners naturally fall into six groups based on their listening patterns — groups that do not map neatly onto genres or demographics but reflect genuine differences in how and when people listen. Those emergent clusters can then inform recommendation algorithms.

School example: A district analyzes attendance patterns across its middle school students without any predetermined groupings. Cluster analysis might reveal three distinct clusters: students with consistently high attendance, students with scattered absences spread across the year, and students with concentrated absences in specific months. Each cluster may call for a different support strategy — and the existence of the third cluster, which a simple average would have hidden, points to seasonal factors worth investigating.

The crucial difference from class description: in cluster analysis, you do not know the groups exist before you start. The algorithm finds them.

4. Association Analysis

Association analysis looks for links between data items that tend to occur together. It is the technique behind the famous retail discovery that customers who buy diapers on Thursday evenings also tend to buy beer — a finding that led stores to place those products near each other despite having no obvious connection.

General example: A grocery chain discovers through association analysis that customers who buy natural peanut butter also tend to buy whole-grain bread, raw honey, and organic apples in the same transaction — even though none of those items are particularly related to each other in obvious ways. The association exists in the data regardless of whether anyone expected it.

School example: A curriculum researcher applies association analysis to course enrollment data across several years. The analysis reveals that students who take an elective in computer science in ninth grade are strongly associated with also enrolling in advanced mathematics courses by eleventh grade — an association that was not visible to counselors making individual scheduling decisions but becomes clear across thousands of enrollment records. This finding might lead to scheduling computer science earlier in students' pathways.

5. Outlier Analysis

Outlier analysis identifies data entries that do not conform to the expected pattern — records that are anomalous relative to everything else in the dataset. Outliers can represent errors in the data, but they can also represent genuinely unusual cases worth investigating.

General example: A credit card company applies outlier analysis to transaction records in real time. A card that has been used exclusively for groceries, gas, and online shopping in Cedar Falls, Iowa suddenly generates a transaction at a luxury hotel in Dubai followed by several large purchases at electronics stores. That pattern is anomalous relative to the card's history — an outlier — and triggers a fraud alert.

School example: A district applies outlier analysis to its gradebook data at the end of each semester. The analysis flags a student whose grades dropped from A's and B's to D's and F's over six weeks — a pattern anomalous enough relative to that student's history and to their peers that it surfaces automatically for a counselor to follow up. Without outlier analysis, this student's change in trajectory might not have been noticed until the semester grades were finalized.

Outlier analysis is also how data quality problems are often detected. A student listed as having 847 absences in a 180-day school year (which we encountered in Topic 5a's data cleaning discussion) would be flagged immediately as an outlier — not a meaningful anomaly, but a data entry error.

6. Sequential Pattern Analysis

Sequential pattern analysis identifies patterns of behavior or events that unfold over time in a consistent order. It asks: do certain sequences of events tend to predict what comes next?

General example: An e-commerce site analyzes the sequence of pages customers visit before making a purchase. Sequential pattern analysis reveals that customers who visit the "reviews" page, then the "compare models" page, then return to the product page are significantly more likely to complete a purchase within the next 24 hours than customers who visit pages in other orders. The site can use this pattern to time promotional offers strategically.

School example: A researcher analyzes longitudinal student data — records spanning multiple years — looking for sequences of academic events that predict later outcomes. Sequential pattern analysis might reveal that students who fail a core course in seventh grade, then retake and pass it in eighth grade, then have high attendance in ninth grade show strong long-term outcomes — while students who follow a different sequence (fail, do not retake, still have high attendance) show much weaker outcomes. The sequence matters, not just the individual events.

A Quick Reference

Here is a summary of the six techniques to help you keep them straight:

Now You Try

For each scenario below, identify which data mining technique is most appropriate and briefly explain your reasoning. Suggested answers are in the answer key.

1. A hospital wants to understand what characteristics patients who are readmitted within 30 days of discharge tend to share, using records from the past five years.
2. An online retailer notices that customers who purchase hiking boots also frequently purchase wool socks, water bottles, and trail maps in the same order, even though these items are in different product categories.
3. A school district analyzes three years of reading assessment scores and discovers — without any prior hypothesis — that students naturally fall into four distinct groups based on their score trajectories over time.
4. A credit card company wants to find transactions that are highly unusual compared to each cardholder's typical spending behavior.
5. A university admissions office wants to find what distinguishes students who complete their degree in four years from those who take six or more years.
6. A public health researcher wants to know whether a particular sequence of symptoms in patient records tends to predict a specific diagnosis being made within the following two weeks.