Advanced Machine Learning
Learn advanced machine learning algorithms through clear, step-by-step explanations and practical Python examples. This module builds Support Vector Machines from the ground up — one small idea at a time — starting from the bias-variance tradeoff that motivates every design choice that follows.
Start here:
- Advanced Machine Learning: The One-Pager — The simplest, one-page summary of every topic in this module. Read this first if you want the big picture.
Recommended Learning Path:
- Bias and Variance — The two ways any model can go wrong: underfitting (bias) vs overfitting (variance). Read this first — every later topic refers back to it.
- SVM Part 1: Hyperplanes and Linear Classification — What a hyperplane is in 2D, 3D, and d-dimensions, and why SVM is a linear model.
- SVM Part 2: Maximal Margin Classifier — Why SVM picks the widest-margin hyperplane, the dot-product and distance-formula refreshers, and support vectors.
- SVM Part 3: Soft Margin Classifier and the Cost Parameter (C) — Slack variables, why the Maximal Margin Classifier is fragile, and how
Ccontrols the bias-variance tradeoff. - SVM Part 4: Kernels and the Kernel Trick — Mapping non-linear data to linear data, feature transformation, the kernel trick, and the
gammaparameter.
Naive Bayes series — probabilistic classification from first principles:
- Naive Bayes Part 1: Probabilistic Classification Intuition — Classification as a probability question, priors and likelihoods, the curse of dimensionality, and why the “naive” assumption saves everything.
- Naive Bayes Part 2: Deriving Bayes’ Theorem — Conditional probability, the full derivation of Bayes’ theorem, and what posterior, likelihood, prior, and evidence mean.
- Naive Bayes Part 3: From Bayes’ Theorem to the Classifier — Extending to multiple features, conditional independence, the argmax decision rule, numerical underflow, and log-space.
- Naive Bayes Part 4: Gaussian Naive Bayes from Scratch — Continuous features, the Gaussian PDF, a hand-worked male/female example, and a full from-scratch implementation compared to scikit-learn.
- Naive Bayes Part 5: Multinomial Naive Bayes — Count data, the multinomial likelihood, Laplace smoothing, and a fully worked spam example.
- Naive Bayes Part 6: Bernoulli Naive Bayes — Binary features, the Bernoulli likelihood, and a fully worked mammal vs non-mammal example.
- Naive Bayes Part 7: Choosing the Right Variant — All three variants compared on the same spam dataset, and why the winner depends on how the feature distribution aligns with each model’s assumption.
- Naive Bayes Part 8: Other Classification Models and Parametric vs Non-Parametric — Compare Naive Bayes with Logistic Regression, Decision Tree, KNN and SVM on the UCI Spambase dataset, and learn the parametric vs non-parametric distinction.
Time Series:
- Time Series - A Friendly Guide — A beginner-friendly introduction to stationarity, trend, seasonality, and additive vs multiplicative decomposition.
Quick reference:
- Advanced Machine Learning: The One-Pager — The simplest, big-picture summary of the whole module.
- Advanced Machine Learning: One-Liners — Every key formula and concept in one easy-to-scan line.
- Naive Bayes — A Friendly Guide — Original single-page overview of Naive Bayes, useful as a quick refresher after the series.
Recommendation Systems:
- Recommendation Systems - A Friendly Guide — How recommendation engines pick the next song, product or movie, from people-who-bought-this to hidden taste maps.
- Recommendation Systems - Part 2 - Framing, Data, Metrics and Baselines — Framing the problem, data contracts, signal strength, evaluation metrics, time-aware splits, leakage prevention and baseline models.
- Recommendation Systems - Part 3 - Models, Ranking and Multi-Modal Systems — Collaborative filtering, matrix factorisation with ALS and BPR, learning-to-rank, negative sampling, feature engineering and multi-modal recommendations.
What You’ll Learn:
- The bias-variance tradeoff and how to diagnose underfitting vs overfitting
- What a hyperplane is, from 2D lines to d-dimensional linear discriminators
- Why maximising the margin leads to better classifiers
- How support vectors define the SVM decision boundary
- Slack variables and the role of the
C(cost) parameter - The kernel trick and when to use linear, polynomial, and RBF kernels
- The
gammaparameter and how it controls boundary flexibility - How each SVM concept maps back to the bias-variance tradeoff
- Bayes’ theorem and how it drives Naive Bayes classification
- When to use SVM vs Naive Bayes vs other classifiers