Reading 3: Reinforcement Learning and the Big Picture

A Fundamentally Different Situation

Supervised and unsupervised learning both start from a dataset — a collection of examples that already exists before training begins. The learning algorithm examines those examples, extracts patterns, and produces a model. The dataset is fixed. The learning happens offline.

Reinforcement learning works differently at a fundamental level. There is no dataset. Instead, there is an agent and an environment. The agent takes actions. The environment responds. Some responses are rewarded; others are penalized. The agent's job is to figure out, through trial and experience, which actions in which situations lead to the most reward over time.

The analogy to human learning is closer here than in the other two categories. A child learning to walk does not study a labeled dataset of successful and unsuccessful steps. She tries, falls, tries differently, falls less often, and gradually develops a skill that works — guided by the continuous feedback of balance, pain, and success. Reinforcement learning is this process, formalized.

The Components of Reinforcement Learning

Every reinforcement learning system has four key components:

• The agent — the system that takes actions and learns. This maps directly to the agent concept from Topic 6a.
• The environment — everything the agent interacts with. In a game, the environment is the game itself. In robotics, it is the physical world. In a recommendation system, it is the platform and its users.
• The reward signal — feedback from the environment after each action. A positive reward indicates a good outcome; a negative reward (penalty) indicates a bad one. The agent does not know in advance which actions are good — it discovers this from the reward signal over time.
• The policy — the strategy the agent develops for choosing actions. A policy maps situations (states) to actions. Learning in reinforcement learning means improving the policy — getting better at choosing actions that lead to more reward.

The goal of reinforcement learning is not to find the action that produces the most reward right now. It is to find the strategy that produces the most reward over time. Sometimes the best move is one that sacrifices immediate reward for a better position later — a distinction that makes reinforcement learning both powerful and difficult.

Where Reinforcement Learning Shines

Games

Games are the natural home of reinforcement learning, for a simple reason: success and failure are unambiguous. Winning is the reward. Losing is the penalty. The rules are fixed. The agent can play millions of games against itself to accumulate experience, and the feedback is immediate and clear.

DeepMind's AlphaGo, which defeated the world champion at the ancient game of Go in 2016, used reinforcement learning. Go has more possible board positions than atoms in the observable universe — far too many for any database approach or explicit rule system. AlphaGo learned by playing millions of games against itself, developing strategies that no human had previously discovered. Two years later, AlphaZero extended the same approach to chess and shogi, starting from nothing but the rules and within hours reaching a level of play that surpassed all previous programs.

Robotics

Teaching a robot to walk is exactly the kind of problem reinforcement learning was designed for. There is no labeled dataset of "correct walking." The physical dynamics are too complex to specify with explicit rules. But a robot can try movements, observe whether it stays upright, receive reward for forward progress and penalty for falling, and gradually develop a stable gait — sometimes one that looks nothing like human walking but works remarkably well.

Recommendation Systems

Recommendation systems — the algorithms that decide what to show you next on a streaming service, social media platform, or news site — often use reinforcement learning. The "reward" is engagement: clicks, watch time, likes. The agent learns which content to recommend to which users in which contexts to maximize that engagement. This is one area where reinforcement learning has been remarkably effective and ethically complicated, since maximizing engagement does not always mean maximizing user wellbeing.

Where Reinforcement Learning Struggles

Reinforcement learning requires a large amount of experience to learn from. In games and simulations, this is fine — a computer can play a million games of chess in an afternoon. In the real world, a million trials may be impractical, expensive, or dangerous. Teaching a self-driving car through pure reinforcement learning in real traffic is not an option. In those situations, supervised learning from human-generated examples, combined with simulation, is typically the more practical approach.

Reinforcement learning also requires a clear, computable reward signal. "Win the game" is clear. "Make this student more engaged in learning" is not — at least not in a form that can be automatically measured and used as a training signal. Many real-world goals are too subtle, too long-term, or too hard to measure for reinforcement learning to apply directly.

The Big Picture: Three Types, Three Questions

We now have the full taxonomy. The three learning types are distinguished by what information is available during training:

This table is your conceptual map for the rest of Week 6 and Week 7. Each row corresponds to a topic, and each topic digs into one specific technique within that learning category. By the time you complete Topics 6d, 6e, and Week 7, you will have seen all three learning types in action, with enough technical depth to explain how each one works and why it is used where it is.

Notice also that these three categories are not a hierarchy from simple to complex. Each solves a fundamentally different problem. Supervised learning requires labels that must be created by humans; reinforcement learning avoids that requirement but needs a computable reward and vast experience. Unsupervised learning requires neither labels nor a reward signal, but can only find structure — it cannot predict specific outcomes. The choice between them is always driven by what kind of data you have and what kind of question you are trying to answer.

What Comes Next

Topic 7a will look at reinforcement learning techniques in depth — specifically hill climbing and genetic algorithms, which are two distinct strategies for searching a solution space guided by a reward signal.

Topic 6e will look at artificial neural networks — the supervised learning approach that powers most of the AI breakthroughs of the last decade, from image recognition to language translation.

Topic 7b will look at unsupervised learning in depth, connecting back to the clustering and pattern discovery concepts from the Week 5 data mining material, and expanding into how unsupervised methods underlie much of what makes modern AI systems powerful.

Topic 7c will look at large language models — how systems like ChatGPT and Claude are trained, what they are actually doing when they generate text, and what their relationship to the learning categories in this topic actually is. (The answer is more complicated than you might expect — LLMs combine elements of all three learning types during different phases of their development.)

Everything in Topics 6d and 6e, and all of Week 7, builds on what you have learned so far. The taxonomy from this topic is the frame. The techniques in Week 7 are what goes inside it.