Advanced Machine Learning

Learn advanced machine learning algorithms through clear, step-by-step explanations and practical Python examples. This module builds Support Vector Machines from the ground up — one small idea at a time — starting from the bias-variance tradeoff that motivates every design choice that follows.

Start here:

Recommended Learning Path:

  1. Bias and Variance — The two ways any model can go wrong: underfitting (bias) vs overfitting (variance). Read this first — every later topic refers back to it.
  2. SVM Part 1: Hyperplanes and Linear Classification — What a hyperplane is in 2D, 3D, and d-dimensions, and why SVM is a linear model.
  3. SVM Part 2: Maximal Margin Classifier — Why SVM picks the widest-margin hyperplane, the dot-product and distance-formula refreshers, and support vectors.
  4. SVM Part 3: Soft Margin Classifier and the Cost Parameter (C) — Slack variables, why the Maximal Margin Classifier is fragile, and how C controls the bias-variance tradeoff.
  5. SVM Part 4: Kernels and the Kernel Trick — Mapping non-linear data to linear data, feature transformation, the kernel trick, and the gamma parameter.

Naive Bayes series — probabilistic classification from first principles:

  1. Naive Bayes Part 1: Probabilistic Classification Intuition — Classification as a probability question, priors and likelihoods, the curse of dimensionality, and why the “naive” assumption saves everything.
  2. Naive Bayes Part 2: Deriving Bayes’ Theorem — Conditional probability, the full derivation of Bayes’ theorem, and what posterior, likelihood, prior, and evidence mean.
  3. Naive Bayes Part 3: From Bayes’ Theorem to the Classifier — Extending to multiple features, conditional independence, the argmax decision rule, numerical underflow, and log-space.
  4. Naive Bayes Part 4: Gaussian Naive Bayes from Scratch — Continuous features, the Gaussian PDF, a hand-worked male/female example, and a full from-scratch implementation compared to scikit-learn.
  5. Naive Bayes Part 5: Multinomial Naive Bayes — Count data, the multinomial likelihood, Laplace smoothing, and a fully worked spam example.
  6. Naive Bayes Part 6: Bernoulli Naive Bayes — Binary features, the Bernoulli likelihood, and a fully worked mammal vs non-mammal example.
  7. Naive Bayes Part 7: Choosing the Right Variant — All three variants compared on the same spam dataset, and why the winner depends on how the feature distribution aligns with each model’s assumption.
  8. Naive Bayes Part 8: Other Classification Models and Parametric vs Non-Parametric — Compare Naive Bayes with Logistic Regression, Decision Tree, KNN and SVM on the UCI Spambase dataset, and learn the parametric vs non-parametric distinction.

Time Series:

  • Time Series - A Friendly Guide — A beginner-friendly introduction to stationarity, trend, seasonality, and additive vs multiplicative decomposition.

Quick reference:

Recommendation Systems:

What You’ll Learn:

  • The bias-variance tradeoff and how to diagnose underfitting vs overfitting
  • What a hyperplane is, from 2D lines to d-dimensional linear discriminators
  • Why maximising the margin leads to better classifiers
  • How support vectors define the SVM decision boundary
  • Slack variables and the role of the C (cost) parameter
  • The kernel trick and when to use linear, polynomial, and RBF kernels
  • The gamma parameter and how it controls boundary flexibility
  • How each SVM concept maps back to the bias-variance tradeoff
  • Bayes’ theorem and how it drives Naive Bayes classification
  • When to use SVM vs Naive Bayes vs other classifiers