Learning Objectives
By the end of this topic, you should be able to:
- Explain the concept of a perceptron: inputs, weights, weighted sum, threshold, and output.
- Given a perceptron with specified weights and threshold, calculate its output for a given set of inputs.
- Explain how positive and negative weights allow a perceptron to model inputs that excite or inhibit a response.
- Describe the layer structure of an artificial neural network: input layer, hidden layer(s), and output layer.
- Explain how a neural network is trained: labeled examples, forward pass, error measurement, weight adjustment, and repetition.
- Explain what deep learning is and how increasing hidden layers changes what a network can learn.
- Explain the tradeoff between the power of deep neural networks and their lack of interpretability compared to decision trees.
Learning Activities
To help you meet the learning objectives, we have prepared a combination of readings, activities, and videos.
Course Readings
These reading were designed to introduce the course topics to an audience of educators. They should be considered "required" and read in order.
- Reading 1 – The Perceptron — from biological neuron to artificial neuron, inputs and weights, a fully worked calculation, and what one perceptron can and cannot do
- Reading 2 – Networks and Learning — layer architecture, how training adjusts weights through repeated prediction and error correction, and ALVINN as a concrete worked example
- Reading 3 – Deep Learning and Why It Matters — what changes with many hidden layers, the capabilities that emerge, real applications, and the power-versus-transparency tradeoff
Supplemental Readings
Some participants find it helpful to read about a topic from a source written for a slightly more technical audience. These supplemental readings cover similar material as the course readings but may not fully align with the course learning objectives. Use them as an optional complement to your study, not a substitute for the course readings.
- Reading: IBM, What is a neural network?
Lesson Videos
These videos support the readings above and may present the material with some deeper connections and worked examples.
- VIDEO: The perceptron: inputs, weights, and calculation.
- VIDEO: A three input perceptron
- VIDEO: Multiple Perceptrons make a Neural Network
- VIDEO: Deep learning with ALVINN
Activity
When this course has a face-to-face component I conduct a hands-on, unplugged activity that gives participants an opportunity to interact with some perceptrons together. While you do not have an opportunity to complete this activity with your peers, I think it is worth your time to read the activity and consider what it would look like both with adults and with your students.
Checking for Understanding, Questions
The Perceptron
-
A perceptron has three inputs with weights 2, −1, and 3, and a threshold of 4. Calculate its output for each of the following input patterns:
- Inputs: 1, 1, 1
- Inputs: 1, 0, 1
- Inputs: 0, 1, 1
- Inputs: 1, 1, 0
- One input to a perceptron has a weight of −5. What does that weight tell you about the relationship between that input and the perceptron's output? Give a real-world example of an input that should have a strong negative weight in a classification system.
Networks and Training
- A neural network is trained to recognize handwritten digits (0–9). During training, it is shown a handwritten 3 and predicts 8. Describe in your own words what happens next. What changes inside the network? What does not change?
- Why does a neural network need hidden layers? What limitation of a single-layer network makes hidden layers necessary?
- ALVINN had a surprisingly simple architecture: 960 inputs, 4 hidden neurons, 30 output neurons. It could nonetheless drive a van at highway speed. What does this tell us about the relationship between network size and network capability?
Deep Learning and Tradeoffs
- A school district is choosing between a decision tree system and a deep neural network for identifying students who may benefit from additional support. The neural network is more accurate. The decision tree is more interpretable. What questions should the district ask before deciding which system to use? What factors make interpretability especially important in this context?
- In Topic 6d, we discussed the overfitting problem for decision trees. Neural networks can overfit too. Given what you know about how neural networks learn, explain in your own words why a network with too many parameters might memorize its training data rather than learn from it.
Checking for Understanding, Answers
You can compare your answers to the following answer key.
Show Answer Key
The Perceptron
-
Weighted sum = (input1 × 2) + (input2 × −1) + (input3 × 3). Output = 1 if weighted sum ≥ threshold (4), else 0.
- Inputs 1,1,1: (1×2) + (1×−1) + (1×3) = 2 − 1 + 3 = 4. 4 ≥ 4 → output = 1
- Inputs 1,0,1: (1×2) + (0×−1) + (1×3) = 2 + 0 + 3 = 5. 5 ≥ 4 → output = 1
- Inputs 0,1,1: (0×2) + (1×−1) + (1×3) = 0 − 1 + 3 = 2. 2 < 4 → output = 0
- Inputs 1,1,0: (1×2) + (1×−1) + (0×3) = 2 − 1 + 0 = 1. 1 < 4 → output = 0
- A weight of −5 means that when this input is active (= 1), it strongly inhibits the perceptron from firing — it subtracts 5 from the weighted sum, making it much harder to reach the threshold. The more active this input is, the less likely the perceptron is to output 1. Real-world example: in a loan approval classifier, a recent bankruptcy on record should have a strong negative weight — its presence strongly argues against approval regardless of other positive factors.
Networks and Training
- The network made an incorrect prediction (8 instead of 3). Next: the error is measured — the difference between the predicted output (high activation on the "8" output neuron) and the correct output (high activation on the "3" output neuron). The weights throughout the network are adjusted slightly to make the "3" prediction more likely and the "8" prediction less likely for this type of input, using an algorithm like backpropagation. What does not change: the network's architecture (number of neurons, layers, and connections) stays fixed; only the values of the weights change.
- A single-layer network can only draw linear decision boundaries in the input space — it can only separate classes that are linearly separable. Many real problems (like XOR, or recognizing handwritten digits) are not linearly separable — no single straight line can divide the cases. Hidden layers allow the network to build up non-linear combinations of features, effectively creating more complex, curved decision boundaries that can separate classes that a single layer could not.
- ALVINN shows that network size alone does not determine capability — a very small network (by modern standards) trained on the right data with the right task can perform impressively. What matters most is whether the training signal (the driving task) contains enough information for the network to extract useful representations. Conversely, simply making a network larger does not automatically improve performance — it needs sufficient high-quality training data and an appropriate learning task.
Deep Learning and Tradeoffs
- Questions the district should ask: Can the neural network explain why a specific student was flagged? If a parent or student challenges the classification, can anyone understand and audit the reasoning? Has the model been evaluated for bias across demographic groups? What happens when the model is wrong — who bears the consequences? Is the accuracy advantage of the neural network large enough to justify the loss of interpretability? Interpretability is especially important in this context because: (1) educational decisions affect real students' opportunities and futures; (2) staff, parents, and students have a right to understand decisions that affect them; (3) bias in the training data could produce systematically unfair classifications that are invisible if the model cannot be inspected.
- A network with very many parameters (weights) has enormous flexibility — enough to memorize every detail of the training examples rather than learning general patterns. During training, the network minimizes its error on the training data; with enough parameters, it can reduce that error to nearly zero by essentially storing each example as a special case rather than finding a rule that generalizes. When new examples arrive, the memorized patterns do not apply, and the network performs poorly. This is the same overfitting problem as for decision trees: too much complexity allows the model to fit noise and coincidence rather than underlying signal.
Extend Your Learning
The following resources go a little deeper on topics we touched on but did not fully explore in the readings. These are entirely optional — none of this material appears on the Competency Demo — but each one is a natural "next question" from something covered this week.
-
Backpropagation
The mathematical algorithm that makes training deep networks practical. Backpropagation efficiently computes how much each weight contributed to the prediction error, allowing all weights to be adjusted in a single pass. This visual article from 3Blue1Brown is one of the clearest introductions available.
Backpropagation — 3Blue1Brown -
Recurrent neural networks and LSTMs
Networks with feedback loops that allow them to process sequential data — text, speech, time series — by maintaining a form of memory across the sequence. The precursor to transformer-based language models. This article from Distill covers the concepts with exceptional clarity.
Attention and Augmented Recurrent Neural Networks — Distill