Part 8: Deep Learning Cheat Sheet — Formulas and Memory Tricks

A quick reference for everything in the Deep Learning series. Every formula and concept uses the spam-detection example.

1. From email to tensor

Real email

“Congratulations! Click here to claim your prize.”

Feature vector

Tensor

email = torch.tensor([3.0, 1.0, 0.0, 0.20, 47.0])

Memory trick: A tensor is a lunchbox with labelled compartments. Each position has a fixed meaning.

2. The artificial neuron

Formula

z = Σ (xᵢ × wᵢ) + b
output = activation(z)

Memory trick

“Shopping basket + bouncer.” Throw all x × w items into a basket, add the bias as the tip, then the activation function decides who gets in.

Worked example

x = [3, 1, 0, 0.20, 47]
w = [0.8, 0.6, -1.2, 2.0, 0.01]
b = -1.0

z = (3×0.8) + (1×0.6) + (0×-1.2) + (0.20×2.0) + (47×0.01) - 1.0
z = 2.4 + 0.6 + 0 + 0.4 + 0.47 - 1.0
z = 2.87

3. Activation functions

For spam detection, the final output uses sigmoid:

probability = torch.sigmoid(z)
# z = 2.87 -> probability ≈ 0.946

4. Forward pass

Input features
    ↓
Hidden layer: z = x @ W + b, then ReLU
    ↓
Output layer: z = h @ W + b, then sigmoid
    ↓
Spam probability

PyTorch version

model = nn.Sequential(
    nn.Linear(5, 4),
    nn.ReLU(),
    nn.Linear(4, 1),
    nn.Sigmoid()
)

email = torch.tensor([[3.0, 1.0, 0.0, 0.20, 47.0]])
probability = model(email)

Memory trick: z = raw score, a = activated score, p = probability.

5. Loss functions

For spam detection, use binary cross-entropy:

criterion = nn.BCELoss()
loss = criterion(prediction, label)

6. Gradient descent

new_weight = old_weight - learning_rate × gradient

gradient = ∂loss / ∂weight

Memory trick: Gradient = direction of downhill. Learning rate = step size.

7. Chain rule

For a path from weight w to loss L:

∂L/∂w = ∂L/∂p × ∂p/∂z2 × ∂z2/∂a1 × ∂a1/∂z1 × ∂z1/∂w

Memory trick: Chain rule = dominoes. Push the first one, trace the effect to the last one.

8. Training loop

for each epoch:
    for each batch of emails:
        1. Forward pass      -> spam probabilities
        2. Compute loss      -> how far from true labels?
        3. Zero gradients    -> optimizer.zero_grad()
        4. Backward pass     -> loss.backward()
        5. Update weights    -> optimizer.step()

Memory trick: Predict -> Measure -> Clear -> Calculate -> Move. Repeat.

9. Tensors and shapes

Memory trick: Shape = address. Rows = examples, columns = features.

10. Tensor operations

11. Dataset and DataLoader

from torch.utils.data import Dataset, DataLoader

class SpamDataset(Dataset):
    def __init__(self, emails, labels):
        self.emails = torch.tensor(emails, dtype=torch.float32)
        self.labels = torch.tensor(labels, dtype=torch.float32)

    def __len__(self):
        return len(self.emails)

    def __getitem__(self, idx):
        return self.emails[idx], self.labels[idx]

loader = DataLoader(SpamDataset(emails, labels), batch_size=32, shuffle=True)

Memory trick: Dataset = single apple. DataLoader = bag of apples.

12. PyTorch parameters

13. Common optimisers

For spam detection, start with Adam:

optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

14. Overfitting vs underfitting

Memory trick: Underfitting = too dumb. Overfitting = too memorised. Good fit = just right.

15. Regularisation

16. Key PyTorch imports

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader, TensorDataset, random_split

17. Twenty memory tricks

1. Tensor = lunchbox with labelled compartments.
2. Rows = examples, columns = features.
3. Weight = volume knob.
4. Bias = starting mood.
5. Raw score = pre-decision number.
6. Sigmoid = thermometer with 0-to-1 scale.
7. ReLU = bouncer (positive in, negative out).
8. Softmax = normalised ranking.
9. Weights = synapses.
10. Gradient = direction of downhill.
11. Learning rate = step size.
12. Epoch = one full pass through the dataset.
13. Batch = one slice of emails.
14. Forward pass = prediction.
15. Backward pass = blame assignment.
16. Loss = how wrong you are.
17. Optimiser = the thing that moves weights.
18. Matrix multiply = inner dimensions must match.
19. Broadcasting = 1 stretches to fit.
20. Dataset = one apple; DataLoader = bag of apples.

18. Common mistakes

Summary

• The spam-detection example is the thread that runs through every tutorial.
• A neuron multiplies inputs by weights, adds a bias, and applies an activation.
• Sigmoid turns raw scores into probabilities. ReLU is the most common hidden activation.
• Training is a loop: predict, measure loss, clear gradients, calculate gradients, update weights.
• Tensors are multi-dimensional arrays. Shape is the most important thing to check.
• Dataset provides one sample; DataLoader provides batches.
• Parameters have requires_grad=True so PyTorch can update them.
• Deep learning mirrors brain ideas but is a mathematical pattern recogniser, not a mind.