Home / PyTorch Intermediate
PyTorch Intermediate
Training done properly: splits and metrics, schedules, regularization and normalization, losses, images and CNNs, embeddings and text, RNNs, transfer learning and data engineering — ending in a full-pipeline capstone. Builds on PyTorch Beginner.
Start learning ❯❯87 lessons · 19 topics · 0/87 completed
Course contents
Work top to bottom — the sidebar is the course. Topics later in the list assume the ones before them.
PyTorch Intermediate Intro
Intermediate · ⏱ 8 min · standalone lesson
Training Discipline
- PyTorch Train, Validation and Test Splits
- PyTorch K-Fold Cross-Validation
- PyTorch The Eval Loop Pattern
- PyTorch Metrics Beyond Accuracy
- PyTorch Thresholds and Decisions
- PyTorch Class Imbalance
- PyTorch Early Stopping
- PyTorch Checkpoint and Resume
Optimizers in Depth
- PyTorch Optimizer State
- PyTorch Parameter Groups
- PyTorch SGD vs Adam in Practice
- PyTorch Gradient Accumulation
- PyTorch Batch Size Effects
Learning Rate Control
- PyTorch Learning Rate Schedules
- PyTorch Warmup and Decay
- PyTorch Finding a Learning Rate
- PyTorch Gradient Clipping
Regularization
- PyTorch Weight Decay
- PyTorch Dropout
- PyTorch Data Augmentation on Tensors
- PyTorch Label Smoothing
- PyTorch Model Capacity
Normalization and Initialization
- PyTorch Why Normalization Layers
- PyTorch Batch Normalization
- PyTorch Layer Normalization
- PyTorch Weight Initialization
- PyTorch Vanishing and Exploding Gradients
Loss Functions in Depth
- PyTorch Writing a Custom Loss
- PyTorch Logits vs Probabilities, the Classic Bug
- PyTorch Combining Losses
- PyTorch Focal Loss
- PyTorch Multi-Label Losses
Images as Tensors
- PyTorch Images as Tensors
- PyTorch Synthetic Image Data
- PyTorch Channels and Colour
- PyTorch Image Batches
- PyTorch Image Transforms by Hand
Convolution
- PyTorch Why Convolution
- PyTorch Convolution by Hand
- PyTorch Conv2d Basics
- PyTorch Padding and Stride
- PyTorch Pooling
- PyTorch Conv Output Shapes
- PyTorch Receptive Fields
- PyTorch Global Average Pooling
A CNN Project
Start: PyTorch The Shapes Dataset ❯Embeddings and Text
- PyTorch Why Embeddings
- PyTorch nn.Embedding
- PyTorch Text as Indices
- PyTorch Bag of Embeddings
- PyTorch Padding Sequences
A Text Classifier Project
- PyTorch The Review Words Dataset
- PyTorch Training a Text Classifier
- PyTorch Reading Text Classifier Errors
Sequences and RNNs
- PyTorch Sequence Data
- PyTorch RNN Basics
- PyTorch Sequence Batching
- PyTorch LSTM Cells
- PyTorch GRU Cells
- PyTorch Bidirectional RNNs
- PyTorch Training a Sine Predictor
Transfer Learning
- PyTorch What Transfer Learning Is
- PyTorch Pretrain on a Synthetic Task
- PyTorch Freeze and Reuse
- PyTorch Fine-Tuning
Data Engineering
- PyTorch Collate Functions
- PyTorch A Dataset from Files
- PyTorch Data Validation
- PyTorch Weighted Sampling
- PyTorch Iterable Datasets
Reproducibility and Debugging
- PyTorch Seeding Everything
- PyTorch Deterministic Mode
- PyTorch Forward Hooks
- PyTorch Hunting a NaN Loss
- PyTorch Checking Gradient Flow
Tuning and Ensembles
- PyTorch Hyperparameter Sweeps
- PyTorch Reading Sweep Results
- PyTorch Training an Ensemble
- PyTorch Ensemble Results, Honestly
Speed on CPU
Start: PyTorch Inference Mode ❯PyTorch Intermediate Capstone
Intermediate · ⏱ 12 min · standalone lesson