Week 3 — Social & Ethical Considerations

Background

This week you studied operating systems: how they evolved, how they manage processes and shared resources, and how they enforce security. These may feel like purely technical topics — but operating systems are among the most consequential software ever written. They mediate access to hardware, data, and networks for billions of people. When they are designed well, they are invisible. When they fail — or are designed with the wrong tradeoffs — the consequences can range from inconvenient to catastrophic.

The scenarios below ask you to think about operating system concepts from a human and ethical perspective. Some involve real events. Some involve ongoing debates with no settled answer. All of them connect directly to the technical content you studied this week.

How to Use These Scenarios

Use the same approach we established in Weeks 1 and 2: read each scenario carefully, write down your initial thoughts using the five guiding questions, and come prepared to discuss in your small group. A full description of the process is available on the SEC scenario discussion guide.

Scenarios

Scenario 1 — The University of Minnesota Linux Incident

This scenario is based on a real event. You can read more here: The Verge — Linux bans University of Minnesota for sending buggy patches in the name of research (2021).

In April 2021, researchers at the University of Minnesota were permanently banned from contributing to the Linux kernel — one of the world's most important open-source operating systems, which runs the majority of servers, Android phones, and embedded systems globally. The reason: it was discovered that the researchers had deliberately submitted patches containing subtle security vulnerabilities as part of a research study designed to show that the Linux kernel review process could be compromised by bad actors.

The researchers argued they were exposing a real and serious weakness in how the Linux community reviewed contributions. The Linux maintainers argued that the researchers had treated a volunteer community as an unwitting test subject and had potentially introduced actual security risks into production code used by millions of systems. The university later apologized. The ban stood.

• Do you believe the researchers acted ethically? Why or why not?
• If you believe their goal was legitimate (exposing a real vulnerability in the review process), was their method appropriate? What could or should they have done differently?
• The Linux kernel is open-source software maintained largely by volunteers. Does that change the ethical calculation compared to, say, submitting flawed patches to a commercial OS vendor?
• The researchers' institution (the University of Minnesota) did not initially take action until public pressure mounted. What responsibility do universities have for the ethical conduct of research that affects outside communities?
• Security researchers routinely use ethically complex methods to expose vulnerabilities. Where should the line be drawn between legitimate security research and harmful experimentation?

Scenario 2 — OS Flaws and Liability

A flaw in a widely-used operating system's privilege level enforcement allows a malicious program to escape its nonprivileged sandbox and gain access to protected memory, including stored passwords and sensitive files. The flaw existed in the OS for three years before it was discovered and patched. During that time, attackers exploited it to compromise thousands of systems, including several at hospitals and financial institutions. Some compromised systems were running versions of the OS that the vendor had designated "end of life" and no longer supported.

This scenario is fictional but draws from patterns seen repeatedly in real incidents — including the EternalBlue/WannaCry ransomware attacks of 2017, which exploited a flaw in Windows and caused billions of dollars in damage, including disruption to the UK's National Health Service.

• To what extent should the OS developer be held responsible for harm caused by a security flaw in their product? Does it matter whether the flaw was the result of a known design tradeoff, an oversight, or deliberate corner-cutting?
• Should OS vendors be legally required to support older versions of their software for a minimum period, given that organizations (particularly hospitals and critical infrastructure) cannot always upgrade quickly?
• Who bears responsibility when an organization knowingly continues to run an unsupported OS because upgrading is expensive or disruptive?
• Analyze the stakeholders in this scenario: OS developer, IT administrators at affected organizations, government regulators, and end users. What responsibility does each bear, and what, if anything, could each have done differently?

Scenario 3 — When the OS Takes Control

This scenario is based on real events. For background on the Boeing 737 MAX crashes, see: The Verge — Boeing 737 Max crashes: what we know about the MCAS software (2019).

Modern commercial airliners and cars are no longer purely mechanical systems operated by human hands. They are computer systems with humans inside them. An enormous amount of what the vehicle does is controlled at the OS and driver code level — monitoring sensors, interpreting data, and in some cases overriding the human operator when the system determines that an unsafe condition exists.

A car may automatically steer to prevent lane departure, or apply the brakes faster than a human could react to avoid a collision. An airplane may take corrective action to prevent an aerodynamic stall if its sensors detect a dangerously steep climb angle.

But systems that trust sensor data to override human judgment create a new category of risk: sensor failure. The Boeing 737 MAX crashes of 2018 and 2019, which killed 346 people, were caused in part by the MCAS (Maneuvering Characteristics Augmentation System) — software that repeatedly pushed the nose of the aircraft down based on faulty data from a single angle-of-attack sensor. In both crashes, pilots fought the system and lost. The software was doing exactly what it was designed to do, based on sensor data that was wrong.

• Explain the fundamental tradeoff between a system that automatically takes corrective action versus one that only displays a warning and leaves the decision to the human operator. When is automated intervention appropriate? When is it not?
• In the Boeing case, the MCAS relied on a single sensor with no redundancy. What does this suggest about the relationship between software design and hardware design? Who should have caught this problem, and when?
• If a sensor malfunction causes an automated system to take dangerous action, who is responsible for the outcome — the software engineers who designed the system, the hardware engineers who designed the sensor, the airline that operated the aircraft, the regulators who certified it, or the pilots who could not override it in time?
• As CS becomes more deeply embedded in safety-critical systems — cars, planes, medical devices, power grids — what ethical obligations do CS professionals have? Does teaching CS carry any of the same obligations?