Neural Networks & Deep Learning

What you’ll be able to do

1. LO1Derive the forward and backward passes of deep fully connected networks in batched matrix form, including softmax cross-entropy, and verify gradients numerically.
2. LO2Implement reverse-mode automatic differentiation over a computational graph, including broadcasting and gradient accumulation, and explain how deep-learning frameworks compute gradients.
3. LO3Build, train, save and evaluate neural networks in PyTorch with correct use of tensors, autograd, modules, data loaders and train/eval modes, reproducibly.
4. LO4Choose activation functions and initialisation schemes from a signal-propagation analysis, and diagnose vanishing, exploding and dead units.
5. LO5Control overfitting in deep networks with weight decay, dropout, early stopping and data augmentation, and justify each choice with validation evidence.
6. LO6Apply batch, layer and group normalisation, derive the batch-norm backward pass, and explain their behaviour in training and inference.
7. LO7Design, implement and analyse convolutional networks: convolution arithmetic and cost, pooling, receptive fields and residual connections.
8. LO8Debug and tune a training run systematically with sanity checks, loss curves, internal statistics, gradient checks and controlled multi-seed experiments.
9. LO9Use embedding layers and learned representations (entity embeddings, autoencoders, linear probes, frozen-feature transfer) and evaluate their quality.

Chapters

12 chapters · each section ends with a short check

1. 01Deep networks as functions: tensors, losses and why depth helps
2. 02Backpropagation in matrix form
3. 03Reverse-mode automatic differentiation: build a tiny engine
4. 04PyTorch fundamentals
5. 05Activations, initialisation and signal propagation
6. 06Regularisation for deep networks
7. 07Normalisation layers
8. 08Training practice and debugging
9. 09Convolution
10. 10Convolutional networks
11. 11Residual networks and training very deep models
12. 12Embeddings and learned representations

Graded labs

1. L1

   Backpropagation by hand: a NumPy MLP for digits

   After chapter 2 · about 9 hours · Optical recognition of handwritten digits · BSD-3-Clause (scikit-learn) · Runs on a CPU

2. L2

   Build a reverse-mode autodiff engine

   After chapter 3 · about 10 hours · Optical recognition of handwritten digits · BSD-3-Clause (scikit-learn) · Runs on a CPU

3. L3

   Making deep networks train: initialisation, normalisation, regularisation and a broken loop

   After chapter 8 · about 10 hours · Optical recognition of handwritten digits · BSD-3-Clause (scikit-learn) · Runs on a CPU

4. L4

   Convolutions from scratch and a small residual network

   After chapter 11 · about 10 hours · Optical recognition of handwritten digits · BSD-3-Clause (scikit-learn) · Runs on a CPU

5. L5

   Embeddings and learned representations

   After chapter 12 · about 10 hours · Synthetic suburban-rail delays · Own work, CC BY 4.0 · Runs on a CPU

Before you start

Where it leads

How it’s taught

You read short sections written for the page. Each ends with a check, and in-browser exercises let you practise. Graded labs run on your own cloud lab machine, CPU-friendly.

No videos. No live classes. No AI tutor.

How it works→