What "Deep" Means
For most of the history of artificial neural networks, researchers worked with shallow architectures: one input layer, one hidden layer, one output layer. ALVINN had this structure. These shallow networks could learn useful things, but they struggled with the most complex perceptual and language tasks — recognizing faces in photographs, transcribing speech, translating between languages.
Deep learning refers to neural networks with many hidden layers — sometimes dozens, sometimes hundreds. The word "deep" refers to the depth of the layer stack, not to any philosophical profundity. Adding depth turns out to matter enormously, and understanding why is the key to understanding why deep learning has transformed the field of AI since roughly 2012.
Why Depth Helps: Hierarchical Features
Consider what a network needs to do to recognize a face in a photograph. The raw input is a grid of pixel brightness values — numbers from 0 to 255, no inherent structure. A face is not a feature that can be detected by looking at individual pixels. It requires building up a hierarchy of concepts:
Each level of this hierarchy corresponds roughly to a layer (or a few layers) in a deep network. Early layers detect simple features directly from raw pixels. Middle layers combine those into more complex patterns. Later layers combine the complex patterns into high-level concepts. The final layer produces a classification.
No human programmed these intermediate representations. The network discovered them on its own during training, because they turned out to be useful for making correct predictions. This is one of the most striking properties of deep learning: the internal representations that emerge from training are often not ones anyone would have thought to design.
What Deep Learning Can Do
The capabilities of deep learning systems are genuinely extraordinary by the standards of AI research from even fifteen years ago. Applications that seemed decades away became practical within a few years of widespread deep learning adoption:
- Image recognition. Modern deep networks recognize objects in photographs with accuracy that matches or exceeds trained human observers in specific domains. Medical imaging systems can detect early-stage tumors in mammograms, diabetic retinopathy in eye scans, and pneumonia in chest X-rays at rates comparable to specialist physicians.
- Speech recognition. The voice recognition on your phone, in your smart speaker, and in customer service systems is powered by deep networks. The dramatic improvement in accuracy that happened around 2010-2012 is almost entirely attributable to deep learning replacing older statistical approaches.
- Machine translation. Google Translate, DeepL, and similar tools use deep networks to produce translations that are far more fluent and accurate than anything possible with earlier rule-based or statistical systems. The improvement in quality over a decade of deep learning is striking to anyone who used machine translation in the 2000s.
- Game playing at superhuman levels. DeepMind's AlphaGo (Topic 6d) used a deep neural network to evaluate board positions, combined with reinforcement learning to improve through self-play. The same approach has been applied to video games, protein structure prediction, and drug design.
- Generative AI. The systems that generate images from text descriptions, compose music, and produce written text — including large language models, which we cover in Topic 7c — are all built on deep neural network architectures.
The Tradeoff: Power Versus Transparency
Deep learning is extraordinarily powerful. It is also nearly impossible to interpret.
A decision tree makes its reasoning explicit. You can print the tree on paper, trace any individual decision, and point to exactly which questions the system asked and why it reached its conclusion. If the decision tree makes a mistake, you can often identify why and fix it.
A deep neural network with millions of weights has no such explanation to offer. The knowledge is distributed across every weight in every layer. No individual weight corresponds to a human-readable rule. When the network classifies an input, you can observe the output, but the path from input to output passes through a sequence of internal transformations that resist human interpretation. This is why deep neural networks are often called black boxes.
For some applications, this is acceptable. If a deep learning image classifier identifies a cat in a photo, the stakes of a wrong answer are low, and accuracy matters more than explanation. But for high-stakes applications — medical diagnosis, loan approval, criminal justice, student assessment — being unable to explain a decision is not just inconvenient. It may be unacceptable.
The more powerful the model, the harder it typically is to explain. The easier it is to explain, the less powerful it typically is. This is not a problem that will be solved by making computers faster. It is a fundamental tension in how these systems work — and navigating it is one of the central challenges of modern AI deployment.
This tension is the subject of a growing field called explainable AI (XAI), which develops techniques for producing human-readable explanations of decisions made by opaque models. The field is active and important — and the fact that it needs to exist tells you something significant about the state of AI today.
What This Means for Teachers
Your students are growing up surrounded by deep learning systems. Every time they use voice recognition, get a social media recommendation, or interact with a customer service chatbot, a deep neural network is involved. Most of them have no idea.
As a teacher, you do not need to be able to train a deep learning model. But you do need to be able to answer a question your students will eventually ask: "How does it actually work?" The answer — layers of neurons, each computing a weighted sum and passing results forward, trained on millions of labeled examples until the weights encode the pattern you want — is now something you can give. It is not magic. It is a very large, very well tuned weighted sum.
And when a student asks "but does it understand?" — you can say: no, not in the way you do. It has learned to produce outputs that look like understanding because it was trained on millions of examples of understanding producing outputs. That is different. And knowing the difference matters.
In Topic 7b, we look at unsupervised learning — neural networks and other techniques that find structure in data without any labels at all. In Topic 7c, we bring everything together to examine large language models, which combine supervised learning, unsupervised learning, and reinforcement learning in ways that produce systems of unprecedented capability — and unprecedented potential for misunderstanding.