Week 5 — Social & Ethical Considerations

Background

This week you learned how data is collected, organized, queried, and mined for patterns. These are powerful capabilities. A well-designed database can give a school district instant access to years of attendance records. A data mining algorithm can surface patterns in student outcomes that no individual teacher or administrator could have noticed on their own. SQL queries can answer questions in seconds that once required weeks of manual report-pulling.

But every capability in this week's topics also carries a shadow. Organized data is easier to query — and easier to misuse. Subschemas can protect privacy — or create a false sense that privacy is being protected. Data mining can identify students who need support — or label them in ways that follow them through their entire school career. The same relational database that makes a district more efficient can make it harder for families to understand what is known about their children, who can see it, and what decisions it is driving.

The scenarios below sit at the intersection of the technical concepts you studied and the real institutions — schools, hospitals, businesses, government agencies — that deploy those concepts at scale. As you read, try to trace the path from a specific technical decision to a human consequence.

How to Use These Scenarios

Use the same approach we established in Weeks 1 and 2: read each scenario carefully, write down your initial thoughts using the five guiding questions, and come prepared to discuss in your small group. A full description of the process is available on the SEC scenario discussion guide.

Scenarios

Scenario 1 — The Score Nobody Asked For

A mid-sized school district partners with an education technology company to implement a "student success platform." The platform ingests data from the district's student information system — attendance records, course grades, disciplinary incidents, standardized test scores, and free-and-reduced lunch status — and applies a data mining algorithm to assign each student a "graduation risk score" updated monthly. Counselors are given a dashboard showing color-coded lists of students: green, yellow, and red.

After the first semester, a high school counselor notices that nearly every student in the red category is from a particular neighborhood and qualifies for free lunch. She also notices that two students she knows well — students she would describe as thriving but navigating genuine financial hardship — have been flagged red. When she asks the vendor how the score is calculated, she is told the formula is proprietary. When she asks which data fields carry the most weight, she receives a vague answer about "predictive factors identified through historical district data."

• What data mining technique is the platform most likely using? What does it mean that the model was trained on "historical district data"?
• If historical patterns in the district's data reflect past inequities — higher discipline rates for certain student groups, for instance — what happens when those patterns are treated as predictive?
• The vendor describes the formula as proprietary. What are the implications of a school making consequential decisions about students based on a process that educators cannot inspect or explain?
• The counselor has direct knowledge that the score misrepresents two specific students. What should she do? What structural barriers might prevent the score from being corrected?
• How does this scenario relate to the distinction between correlation and causation that came up in Topic 5a?

Scenario 2 — The Subschema That Wasn't

A regional hospital network consolidates its patient records into a new enterprise database system. During implementation, the IT team designs subschemas carefully: billing staff can see insurance and payment data but not clinical notes; floor nurses can see current medications and allergies but not mental health records; and so on. The database administrator receives praise for a thoughtful, layered design.

Six months after launch, the network signs a contract with a third-party analytics firm to study patterns in patient readmission rates. To facilitate the analysis, an IT technician exports a large dataset to a flat file and transfers it to the firm's servers. The export process pulls from multiple relations and includes fields that would have been invisible to most hospital staff under the subschema restrictions. The analytics firm's contract says the data will be used only for "quality improvement research," but the firm is later acquired by a health insurance company.

• The hospital's subschema design protected patient data within the institution. What happened to that protection the moment the data was exported to a flat file?
• The technician who performed the export was likely following a reasonable workflow. Where in the process should a check have existed that didn't?
• The contract language — "quality improvement research" — was technically accurate at the time it was signed. What does this tell us about the limits of contractual data governance?
• From a database design perspective, what is the difference between controlling access to a live system and controlling what happens to data once it leaves that system?
• As a teacher, you handle student data every day. What parallels do you see between this hospital scenario and the way student data moves through your own school or district?

Scenario 3 — The Loyalty Program You Didn't Know You Joined

This scenario is based on real events. The Target pregnancy prediction story was first reported by Charles Duhigg in the New York Times in 2012. You can search for "Target pregnancy prediction New York Times Duhigg" to find the original reporting, or read this summary.

A large grocery chain operates a rewards card program. Customers scan their card at checkout in exchange for discounts on selected items. The chain stores every transaction — every product, quantity, price, date, time, and store location — linked to the cardholder's account. Over years of operation, the database grows to tens of millions of records. The chain hires a data science team to mine the purchase data.

The team applies association analysis and discovers that customers who regularly buy certain combinations of products — unscented lotion, vitamin supplements, and specific over-the-counter medications — are statistically likely to be pregnant, often before the customer has shared that information with anyone. The chain begins sending targeted coupons to these customers. In one widely reported case, a family receives pregnancy-related coupons addressed to their teenage daughter before the family is aware of her pregnancy.

The chain's legal team reviews the situation and determines that no laws were broken. The customer voluntarily enrolled in the rewards program. The terms of service, which almost no one reads, permitted the use of purchase data for "personalized marketing purposes."

• The data mining technique here is association analysis. At what point did the inference cross a line — if it did?
• The family's daughter did not consent to her purchases being used to infer personal health information. Does it matter that a parent's card was used for the transaction?
• "No laws were broken" is a conclusion, not an ethical evaluation. What is your ethical evaluation?
• The terms of service permitted this use. Is meaningful consent possible when terms of service are this broad and this rarely read? What would meaningful consent look like?
• This scenario involved a grocery store, but data mining is increasingly used in educational technology platforms. What parallels exist between a loyalty program database and the data collected by an adaptive learning platform or a school-issued device?