Part 18: Activation and Output Functions — A Friendly Guide
A beginner-friendly guide to the most common activation and output functions in deep learning, with formulas, use cases, and a decision table.
Welcome to My Technical Handbook
Hi! I'm Rilov Paloly Kulankara, and this is where I share everything I'm learning in the world of technology. As a software engineering leader, I'm constantly exploring new concepts in data engineering, cloud architecture, security, and scalability. This handbook is my way of documenting these learnings in simple, beginner-friendly guides with diagrams and real-world examples. Whether you're a seasoned engineer or just starting out, I hope you find these notes helpful on your own learning journey!
All Topics
Part 10: Advanced Convolutional Neural Networks - A Friendly Guide
A simple next step after the beginner CNN guide. Explains advanced CNN ideas in plain English, with diagrams throughout — deeper networks and the degradation problem, overfitting, dropout, batch normalization, residual connections, global average pooling, 1x1 bottleneck convolutions, CNN backpropagation through filters, feature maps, ReLU, and pooling, transfer learning, data augmentation, and modern CNN architecture patterns.
SVM Part 4: Kernels and the Kernel Trick
Step four of the SVM series: how kernels let a linear model separate non-linear data, why the kernel trick avoids expensive feature transformations, and how to choose between linear, polynomial, and RBF kernels.
Naive Bayes Part 3: From Bayes' Theorem to the Naive Bayes Classifier
Part 3 of the Naive Bayes series: extend Bayes' theorem to multiple features, add the conditional independence assumption, and derive the final Naive Bayes decision rule with argmax.
Naive Bayes Part 4: Gaussian Naive Bayes from Scratch
Part 4 of the Naive Bayes series: build a Gaussian Naive Bayes classifier from scratch, compare it to scikit-learn, and interpret priors, likelihoods, and posteriors.
Naive Bayes Part 5: Multinomial Naive Bayes
Part 5 of the Naive Bayes series: the Multinomial variant for count data, the multinomial likelihood formula, Laplace smoothing, and a fully worked spam example.
Naive Bayes Part 6: Bernoulli Naive Bayes
Part 6 of the Naive Bayes series: the Bernoulli variant for binary yes/no features, its likelihood formula, and a fully worked mammal vs non-mammal example.
Naive Bayes Part 7: Choosing the Right Variant
Part 7 of the Naive Bayes series: compare Gaussian, Multinomial, and Bernoulli Naive Bayes on the same spam dataset, and learn why the best variant depends on how the feature distribution aligns with each model's assumption.
Naive Bayes Part 8: Other Classification Models and Parametric vs Non-Parametric
Part 8 of the Naive Bayes series: compare Naive Bayes with Logistic Regression, Decision Tree, KNN and SVM on the UCI Spambase dataset, then learn the conceptual distinction between parametric and non-parametric models.
Advanced Machine Learning: One-Liners
Every key formula and concept in the Advanced Machine Learning module, reduced to one easy-to-scan line.
Advanced Machine Learning: The One-Pager
The simplest one-page guide to every topic in the Advanced Machine Learning module: bias and variance, SVM, and the full Naive Bayes series.
26. Anomaly Detection - A Friendly Guide
A beginner-friendly guide to anomaly detection. Learn how to find unusual data points using isolation forest, one-class SVM, and statistical methods. Covers fraud detection, network intrusion, and quality control with full Python examples.
25. Association Rules and Apriori - A Friendly Guide
A beginner-friendly guide to association rules and the Apriori algorithm. Learn how to find "customers who bought X also bought Y" patterns in transaction data, with support, confidence, lift explained and full Python examples.
Bias and Variance - A Friendly Guide
A step-by-step, beginner-friendly introduction to bias and variance — the two kinds of error every model can make. Explains underfitting vs overfitting, the bias-variance tradeoff, and how to spot each problem in practice, before moving on to more advanced models like SVM.
Part 12: Building a CNN End-to-End — CIFAR-10 with PyTorch
A complete hands-on walkthrough of building, training, and evaluating a CNN on the CIFAR-10 dataset using PyTorch. Covers loading and preprocessing the dataset, defining a CNN architecture, writing the training loop with validation, and evaluating with accuracy, confusion matrix, and classification report.
Part 13: CNN Applications — Image Classification, Object Detection, Segmentation, Facial Recognition, and OCR
An overview of the main real-world applications of CNNs — image classification, object detection, image segmentation, facial recognition and analysis, and optical character recognition (OCR). Explains what each task is, how CNNs are used, key architectures, and real-world examples.
Part 11: CNN Training Pipeline, Transfer Learning, and Visualization — A Friendly Guide
Covers the full CNN model design pipeline — from data collection and preprocessing through training — then explains when to train from scratch versus use transfer learning or fine-tuning, and finally shows how to visualize what a CNN has actually learned using Grad-CAM and activation maps.
Chapter 1: Exploratory Data Analysis
Learn the fundamentals of Exploratory Data Analysis (EDA) - understanding your data through visualization, summary statistics, and pattern discovery.
Chapter 2: Inferential Statistics and Hypothesis Testing
Learn how to make predictions about populations from samples, understand confidence intervals, and test hypotheses using statistical methods.
Chapter 3: Introduction to Hypothesis Testing
Master the fundamentals of hypothesis testing - learn the 5-step process, understand p-values, and avoid common errors in statistical testing.
Chapter 4: Hypothesis Testing Methods
Comprehensive guide to statistical hypothesis testing methods - single sample tests, two sample tests, and proportion tests with practical examples.
Chapter 4.1: Single Sample Tests
Learn single sample hypothesis tests - z-tests and t-tests for comparing one sample to a claimed value, with practical examples and Python code.
Chapter 4.2: Two Sample Tests
Master two sample hypothesis tests - independent and paired t-tests for comparing two groups, with real-world examples and Python implementations.
Chapter 4.3: Proportion Tests
Learn proportion tests for testing percentages and proportions - one-sample and two-sample proportion tests with practical examples and Python code.
Classical Text Generation - A Friendly Guide
A beginner-friendly guide to classical text generation. Learn how computers generate language using N-gram models, Markov chains, probability, smoothing techniques, and how to evaluate quality with perplexity—before the era of neural networks.
Classical Text Generation Part 2 - Summarisation and Translation
Learn how classical NLP systems generate sentences, automatically summarise documents, and translate between languages—before neural networks. Covers controlled sentence generation, extractive summarisation, and rule-based and statistical machine translation.
Part 7: Cognitive Analogies - Brain-inspired Learning
Learn how deep learning mirrors ideas from the brain, using the spam-detection example throughout. Covers pattern recognition, memory, attention, generalisation, and learning from mistakes.
14. Ensembles - Part 4 - Voting Classifier Example
A complete worked example of a soft-voting ensemble on the breast cancer dataset using Logistic Regression, Random Forest and Gradient Boosting.
Computer Vision - Part 1: Image Processing Fundamentals
A beginner-friendly introduction to digital images as numerical grids. We cover pixels, color models, image shapes, matrix operations, filters, convolution and feature extraction.
Computer Vision - Part 2: Introduction
A beginner-friendly introduction to computer vision: what it is, why it is hard, the standard pipeline, and how it connects to deep learning and image processing.
Computer Vision - Part 3: CNNs for Computer Vision
A very simple, picture-heavy primer on convolutional neural networks written specifically for computer vision, explained the way you'd explain it to a curious kid: learned filters, feature maps, pooling, the hierarchy of features, and the idea of a pretrained CNN backbone, just enough to understand the object detectors covered later in this series.
Computer Vision - Part 4: Object Detection Basics
A beginner-friendly guide to object detection and localisation: bounding boxes, coordinate formats, the detection pipeline, IoU, precision, recall, and mAP.
Computer Vision - Part 5: Loss in Object Localisation
A simple guide to loss functions used for bounding-box regression, from L1/L2 and Smooth L1 to IoU, GIoU, DIoU, and CIoU, with formulas and code examples.
Computer Vision - Part 6: Region-Based Object Detectors
A beginner-friendly guide to region-based object detectors: R-CNN, Fast R-CNN, Faster R-CNN, the Region Proposal Network, fully convolutional design, and the limits of two-stage detection.
Computer Vision - Part 7: Anchor Boxes
A beginner-friendly guide to anchor boxes: why they exist, how they are placed and matched to ground truth, and how they are used in Faster R-CNN, YOLO, and SSD.
Computer Vision - Part 8: One-Stage Detectors — YOLO and SSD
A beginner-friendly guide to one-stage object detectors, with detailed explanations of YOLO and SSD: how they work, how to decode their outputs, and how non-maximum suppression produces the final boxes.
Computer Vision - Part 9: YOLO Evolution and YOLO11 Demo
A beginner-friendly history of YOLO versions and a practical walkthrough of object detection with YOLO11 using Ultralytics and Python.
Part 9: Convolutional Neural Networks (CNNs) - A Friendly Guide
A zero-background introduction to Convolutional Neural Networks. Starts with a simple cat-vs-dog photo example and builds up, one idea at a time, through convolution, activation, feature maps, padding, and pooling — then goes further into the exact output-size and parameter-count formulas, finishing with a full layer-by-layer walkthrough of the real VGG16 architecture.
Part 4: Data Handling with Dataset and DataLoader - A Friendly Guide
Learn how to load, batch, shuffle, and iterate over spam-detection data in PyTorch using torch.utils.data.Dataset and DataLoader. Uses the same email example throughout.
08. Decision Trees - Part 1 - Foundations
A simple beginner friendly guide to decision trees. Part 1 covers what they are, why we use them, and how they actually work step by step using real life examples.
09. Decision Trees - Part 2 - Measures of Impurity
Part 2 of the decision trees tutorial. We explain in very simple terms what impurity means, how to measure it using Gini and entropy, and how the algorithm decides which question to ask using information gain.
10. Decision Trees - Part 3 - Training and Demonstration
Part 3 of the decision trees tutorial. We walk through the full training process, talk about overfitting and pruning, discuss real world advantages and disadvantages, and then build a working decision tree in Python on a real dataset.
Part 8: Deep Learning Cheat Sheet - Formulas and Memory Tricks
A one-page reference for every deep learning formula and concept from the tutorial series, all using the spam-detection example. Each formula has a memory trick.
11. Ensembles - Part 1 - Foundations of Ensemble Learning
A simple introduction to ensemble learning. Why combining many weak models beats relying on one strong model. The wisdom of crowds, bias and variance, and the two big families of ensembles.
12. Ensembles - Part 2 - Bagging and Random Forests
Part 2 of the ensembles tutorial. We explain bootstrap sampling, build up to the Random Forest algorithm step by step, and walk through full hyperparameter tuning using number of trees, maximum depth, and grid search with cross validation.
13. Ensembles - Part 3 - Boosting and Modern Methods
Part 3 of the ensembles tutorial. We explain boosting from the ground up, walk through AdaBoost and Gradient Boosting in plain language, meet the modern champions like XGBoost and LightGBM, and finish with practical advice on when to use each method.
Part 14: Faster R-CNN and Region Proposal Networks (RPN) - A Friendly Guide
A zero-background, kid-friendly walkthrough of the actual Faster R-CNN paper (Ren, He, Girshick, and Sun, 2015). Explains why finding objects in a photo is harder than just naming what's in it, how R-CNN and Fast R-CNN worked and where they got stuck, and how the paper's big idea, the Region Proposal Network, lets one single network both find and name objects almost for free. Diagrams throughout, plus a real example straight from the paper.
Part 6: Forward and Backward Propagation - A Friendly Guide
Learn exactly what happens during forward and backward propagation, using a two-layer spam-detection model. Worked numerical example with the chain rule included.
Part 2: From Machine Learning to Deep Learning - A Friendly Guide
The direct continuation of Part 1. Part 1 built the spam model with handcoded weights. Part 2 answers the natural next question — how does a model actually learn those weights? Covers labels, loss, backpropagation, and the complete training loop using the same spam example.
24. Gaussian Mixture Models - A Friendly Guide
A beginner-friendly guide to Gaussian Mixture Models (GMMs). Learn how probabilistic clustering differs from K-Means, what the EM algorithm does, and how to fit GMMs in Python for soft cluster assignments.
16. Hierarchical Clustering
A complete beginner friendly guide to hierarchical clustering. We cover the agglomerative algorithm step by step, the different linkage methods, how to read a dendrogram, practical considerations, and a full Python demonstration.
Hyperparameters and Tuning - A Friendly Guide
A friendly guide to hyperparameters: what they are, how they differ from learned parameters, and how to tune them with grid search, random search, early stopping, and a careful eye on budget and leakage.
15. K-Means Clustering
A complete beginner friendly guide to K-Means Clustering. We cover unsupervised learning, how K-Means actually works step by step, picking the right number of clusters, common pitfalls, and a full Python demonstration.
17. K-Nearest Neighbours
A complete beginner friendly guide to K-Nearest Neighbours (KNN). We cover how it works, weighted aggregation, picking K, key practical considerations, and a full Python demonstration for both classification and regression.
Part 16: LSTM and GRU - Solving the Memory Problem - A Friendly Guide
A beginner-friendly guide to LSTM and GRU — the improved versions of RNN that can remember things from much longer ago. Learn why basic RNNs forget, how LSTM gates solve this, and how to use both in PyTorch.
Lakehouse Serving Architecture: Bronze, Silver, Gold, and Serving Projections
A practical architecture guide explaining why Bronze/Silver/Gold describe data quality while Serving describes consumption. Covers how to keep Gold as the authoritative Iceberg source of truth and publish workload-specific serving projections to Trino, StarRocks/ClickHouse, and Redis/Scylla, plus how to expose governed metrics to AI agents through a semantic layer and MCP server — with Mermaid diagrams throughout.
Lexical Processing - A Friendly Guide
A beginner-friendly guide to Lexical Processing in NLP. Learn how computers break down text through tokenization, text normalization, stopword removal, stemming, lemmatization, and spell correction—with simple explanations and Python examples.
04. Linear Regression - Advanced Topics
Deep dive into Linear Regression assumptions, challenges, and advanced techniques - scaling, feature engineering, handling violations, and regularization explained simply.
05. Linear Regression - Expert Topics
Master Linear Regression with expert-level topics - categorical encoding, deep multicollinearity, influential points, cross-validation, polynomial features, and real-world techniques.
03. Linear Regression - Introduction
Complete beginner's guide to Linear Regression - learn to predict continuous values with real-world examples and Python code.
06. Logistic Regression - Introduction
Master Logistic Regression from zero to hero - the most popular algorithm for classification. Beginner-friendly explanations with real examples and Python code.
31. ML Algorithm Intuition Flashcards
One flashcard per algorithm. The core idea, an everyday analogy, when to use it, and a tiny code snippet. Perfect for review, revision, or refreshing your memory right before an interview.
29. ML Common Pitfalls and Debugging Guide
A symptom to cause to fix troubleshooting guide for the most common machine learning problems. When your model behaves weirdly, look up the symptom here and find out what is going wrong.
33. ML End-to-End Worked Project - Titanic
A complete end-to-end ML project on the Titanic dataset, from raw CSV to final evaluation. We do data exploration, cleaning, feature engineering, model comparison, hyperparameter tuning, and final test set evaluation - showing exactly how all the pieces from the previous tutorials fit together.
30. ML FAQ - What is the Difference Between...
A frequently asked questions page answering every common 'what's the difference between X and Y' question in machine learning. Quick, clear, side-by-side comparisons of the concepts that confuse beginners most.
28. ML Glossary - Plain English Dictionary
An A-to-Z plain English dictionary of every machine learning term used in this handbook. No equations, no jargon defined with more jargon. Just clear short explanations for every concept you will meet.
27. ML Quick Reference - Libraries, Metrics and Cheat Sheet
A single-page quick reference for every ML topic in this handbook. Libraries to import, classes to use, key hyperparameters, evaluation metrics with formulas, and a master cheat sheet for picking the right tool and the right metric.
32. ML Visual Algorithm Comparison
A side-by-side visual tour of how every classifier carves up the same 2D dataset. See exactly why linear models draw straight lines, decision trees make boxes, KNN makes wavy regions, and ensembles smooth things out.
02. ML in Plain English - A Friendly Tour
A friendly story-style tour of every machine learning technique covered in this handbook. No math, no code, no jargon. Just a clear, plain English explanation of what each tool does and when you would actually use it in real life.
01. Machine Learning - Introduction
A friendly, story-style introduction to Machine Learning. What it is, how it differs from regular programming, the three big types, the full ML pipeline, and the key ideas every beginner should know. With diagrams.
Machine Learning with NLP - A Friendly Guide
A beginner-friendly guide to how machine learning works with text. Learn how computers understand language through simple, real-world examples—no heavy math, just clear explanations of sentiment analysis, spam detection, named entity recognition, and more.
Mathematics 01: Probability Foundations — Questions, Formulas, and Answers
Learn probability from zero using simple questions, formulas, plain-English explanations, and worked answers.
Mathematics 02: Counting for Probability — Permutations and Combinations
Count possible outcomes using the multiplication rule, permutations, and combinations before calculating probability.
Mathematics 03: Conditional Probability — Questions, Formulas, and Python
Learn conditional probability using simple class examples, complete formulas, input validation, and a Python solution.
Mathematics 04: Independence and Bayes' Theorem — Questions, Formulas, and Answers
Learn whether events affect each other and use Bayes' theorem to update probability after new evidence.
Mathematics 05: Random Variables and Distributions — Questions, Formulas, and Answers
Understand random variables, probability distributions, expected value, variance, and standard deviation through simple questions.
Mathematics 06: Advanced Probability — Joint Distributions, Covariance, LLN, and CLT
Connect probability ideas using joint distributions, covariance, conditional expectation, the law of large numbers, and the central limit theorem.
Model Building and Validation - Telco Churn Example
A practical walkthrough of building and validating a churn model: loading data, building a scikit-learn pipeline, using holdout, K-fold, and stratified cross-validation, and avoiding data leakage.
Model Evaluation - A Friendly Guide
A friendly guide to evaluating machine learning models: train-test split, validation sets, cross-validation, metrics, and how to avoid common traps.
NLP Cheat Sheet - Key Terms and Concepts
A quick-reference cheat sheet covering every important NLP term and concept from the full tutorial series. Organised by topic—from tokenisation to machine translation—with plain-English definitions and one-line memory aids.
NLP Libraries - Python Toolkit Reference
A complete reference guide to every Python library used across the NLP tutorial series. Learn what each library does, how to install it, and which functions to use for each NLP task—with ready-to-run code examples.
18. Naive Bayes - A Friendly Guide
A complete beginner-friendly guide to Naive Bayes. Learn how probability drives classification, why the "naive" assumption works in practice, and build a spam detector step by step in Python.
Naive Bayes Part 1: Probabilistic Classification Intuition
Part 1 of the Naive Bayes series: why classification is fundamentally a probability question, and how we use evidence to update our beliefs about which class a sample belongs to.
Naive Bayes Part 2: Deriving Bayes' Theorem from Conditional Probability
Part 2 of the Naive Bayes series: derive Bayes' theorem from the definition of conditional probability, and understand what each term means.
Natural Language Processing - A Friendly Introduction
A beginner-friendly introduction to Natural Language Processing. Learn what NLP is, where it's used across industries, the stages of the NLP pipeline, and the different approaches to building NLP systems—using simple stories and real-world examples.
20. PCA and Dimensionality Reduction - A Friendly Guide
A complete beginner-friendly guide to PCA and dimensionality reduction. Learn why too many features hurt models, how PCA finds the directions of maximum variance, and how to apply it in Python step by step.
Part 5: Parameters in PyTorch - A Friendly Guide
Learn how PyTorch creates, tracks, and updates learnable parameters for a spam-detection model. Understand weights, biases, requires_grad, and how to inspect and manage model parameters.
Part 15: RNN - Recurrent Neural Networks - A Friendly Guide
A beginner-friendly introduction to Recurrent Neural Networks. Learn why normal neural networks cannot handle sequences, how an RNN reads data step by step, and how to build one in PyTorch using simple everyday examples.
Part 17: RNN Use Cases - Time Series, Sequence Classification and Labelling - A Friendly Guide
A beginner-friendly guide to practical RNN applications. Learn how to use LSTM for time series forecasting, sequence classification, and sequence labelling with clear step-by-step Python examples.
Recommendation Systems - A Friendly Guide
A beginner-friendly guide to how recommendation systems work: people-who-bought-this, people-like-you, hidden taste maps, and learning from clicks instead of star ratings.
Recommendation Systems - Part 2 - Framing, Data, Metrics and Baselines
The foundations of recommendation systems: how to frame the problem, what data to collect, how to build data contracts, how to split data without leaking the future, and the most useful offline evaluation metrics.
Recommendation Systems - Part 3 - Models, Ranking and Multi-Modal Systems
The main models and techniques: collaborative filtering, implicit matrix factorisation with ALS and BPR, learning-to-rank, negative sampling, feature generation and multi-modal recommendation.
07. Regularization - Complete Guide
Master Regularization from zero - Ridge, Lasso, Elastic Net explained simply with analogies, math, and Python code. Stop overfitting forever!
23. Reinforcement Learning - A Friendly Guide
A beginner-friendly guide to reinforcement learning. Learn how an agent learns by trial and error using rewards and punishments, how Q-learning works, and how RLHF is used to train language models like ChatGPT.
SVM Part 1: Hyperplanes and Linear Classification
Step one of the SVM series: what a hyperplane is in 2D, 3D, and beyond, and why SVM is considered a linear model. Builds the mathematical foundation needed for margins and kernels in the topics that follow.
SVM Part 2: Maximal Margin Classifier
Step two of the SVM series: why SVM picks the hyperplane with the widest margin, how the margin is measured using the dot product and distance formula, and why the maximal margin classifier is fragile on noisy real-world data.
SVM Part 3: Soft Margin Classifier and the Cost Parameter (C)
Step three of the SVM series: how the Soft Margin Classifier fixes the fragility of the Maximal Margin Classifier using slack variables, and how the C (cost) parameter controls the bias-variance tradeoff for SVM.
22. Self-Supervised Learning - A Friendly Guide
A beginner-friendly guide to self-supervised learning — how models train themselves on unlabelled data by predicting parts of the input from other parts. Covers masked prediction, contrastive learning, and the role of SSL in GPT and BERT.
Semantic Processing Part 1 - Word Meaning and Disambiguation
A beginner-friendly guide to how computers understand word meaning. Covers lexical semantics, word sense disambiguation, co-occurrence models, and how to measure word similarity using statistics.
Semantic Processing Part 2 - Entities, Roles, and Coreference
Learn how computers identify who is doing what to whom in a sentence. Covers Semantic Role Labelling, Named Entity Recognition, IOB tagging, Conditional Random Fields, and Coreference Resolution—with simple explanations and Python examples.
21. Semi-Supervised Learning - A Friendly Guide
A beginner-friendly guide to semi-supervised learning — the technique that uses a small amount of labelled data and a large amount of unlabelled data together. Covers self-training, label propagation, and GANs with Python examples.
14. Similarity and Distance Metrics
A complete beginner friendly guide to the metrics machine learning uses to decide whether two points are similar or different. We cover distance-based, angle-based, correlation-based, and set-based metrics with simple examples and Python code.
19. Support Vector Machines - A Friendly Guide
A complete beginner-friendly guide to Support Vector Machines. Learn how SVMs find the best boundary between classes, what the kernel trick does, and how to use SVMs in Python for real classification tasks.
Syntactic Processing Part 1 - POS Tagging and Shallow Parsing
Learn how computers identify word types (nouns, verbs, adjectives) and group them into meaningful phrases. A beginner-friendly guide to Part-of-Speech tagging and shallow parsing with Python examples.
Syntactic Processing Part 2 - Context-Free Grammar and Parsing
Learn how computers understand sentence structure through grammar rules, constituency parsing, and dependency parsing. A beginner-friendly guide to analyzing how words relate to each other.
Syntactic Processing Part 3 - Tree Structures and Grammatical Agreement
Learn how to visualize sentence structure with parse trees and ensure grammatical correctness through agreement checking. The final part of our syntactic processing series.
Part 3: Tensors and Tensor Operations - A Friendly Guide
Learn tensors and tensor operations using the same spam-detection example. Covers feature vectors, batches of emails, weights, matrix multiplication, broadcasting, and reshaping. Builds directly on the training loop from Part 2.
Text Representation and Unsupervised Modelling - A Friendly Guide
Learn how computers turn raw text into numbers, and how to discover hidden topics in large collections of documents—without any labelled data. Covers Bag-of-Words, TF-IDF, Word Embeddings, and Topic Modelling with NMF.
Time Series - A Friendly Guide
A beginner-friendly guide to time series: what it is, how to spot changing mean and variance, what stationarity means, and how to decompose a series into trend, seasonality, and residuals.
Time Series Part 2 - Simple Models and Smoothing
A beginner-friendly guide to the simplest time-series forecasting models: naive, mean, drift, seasonal naive, simple exponential smoothing, Holt's method, and Holt-Winters. Plus how the parameters are chosen.
Time Series Part 3 - Autoregressive Models and Stationarity
A beginner-friendly guide to autoregressive models, stationarity tests, handling non-stationarity, and autocorrelation measures (ACF and PACF).
Time Series Part 4 - ARMA, ARIMA and SARIMA
A beginner-friendly guide to Moving Average, ARMA, ARIMA, and SARIMA models, with a worked example using a simple synthetic sales dataset.
Time Series Part 5 - Practical Forecasting
Practical forecasting ideas explained in plain English: white noise, forecast horizons, rolling windows, walk-forward validation, forecast intervals, and when to use LSTM, GRU, Transformer and TFT for time series.
Part 1: Understanding Neurons and Neural Networks - A Friendly Guide
Learn how a neural network decides whether an email is spam by turning real-world information into numbers, multiplying by weights, adding a bias, and applying an activation function. Uses a single, concrete example throughout.
Saturday Morning Coffee ☕ Thoughts on Agentic AI
Why Agentic AI isn't failing because it's weak it's failing because we're asking it to do the wrong job. Agents should orchestrate workflows, not replace them.
Case Study: How Airbnb Solved Metric Chaos (Part 1 - The Problem)
How Airbnb discovered their biggest data problem wasn't big data — it was getting everyone to agree on what "bookings" actually means.
Case Study: How Airbnb Solved Metric Chaos (Part 2 - The Solution)
How Airbnb built Minerva — a platform that turned 10,000 inconsistent metrics into a single source of truth, saving 60% of engineering time.
Skills vs MCP vs Agents: The Future of AI (Explained Simply)
How Apache Iceberg Actually Works (And Why It's Perfect for AI)
A user-friendly technical guide to Apache Iceberg's architecture - how it works under the hood, why it's different from traditional tables, and why it's the go-to choice for AI/ML workloads.
Avro Deep Dive — Schema Evolution for Streaming Data
How Apache Avro enables schema evolution, efficient serialization, and seamless streaming data with embedded schemas.
Scaling Your API Part 3: Choosing the Right Database
Learn how to choose the right database for your API — from handling thousands to millions of requests per second.
Data File Formats — The Complete Guide
A beginner-friendly guide to every data file format from CSV to Delta Lake with simple explanations and visual examples.
Database Types — A Complete Guide
Understand different database types (RDBMS, Columnar, NoSQL, Vector) with simple explanations, popular options, and when to use each.
Delta Lake Deep Dive — ACID Transactions on Data Lakes
How Delta Lake brings ACID transactions, time travel, and schema evolution to Parquet files on data lakes.
Dimensional Modeling — A Beginner's Guide
Learn the fundamentals of Kimball dimensional modeling with simple analogies, diagrams, and practical examples.
Hashing vs Encryption — Quick Comparison
A beginner-friendly guide comparing hashing and encryption with simple analogies and Python code examples.
How Iceberg Query Engines Work Internally — Trino, Spark & the REST Catalog
A deep dive into how query engines like Trino and Spark execute queries against Iceberg tables — from REST Catalog API calls to reading Parquet files from S3.
Kubeflow Spark Operator - Getting Started
Learn how to run Apache Spark on Kubernetes using the Spark Operator. This guide covers the basics: what it is, how it works, and how to submit your first Spark job without manual spark-submit commands.
Kubeflow Spark Operator - Enterprise & Multi-Tenant
Enterprise-grade Spark on Kubernetes: multi-tenant architecture, centralized logging and monitoring, self-service job submission portals, cost tracking, and security best practices for running Spark at scale.
Part 2 - LangChain Essentials: Building Blocks of AI Applications
Learn the core LangChain building blocks - chat models, prompting patterns, structured outputs, and chaining - explained simply.
Part 1 - LangChain Foundations: Understanding LLMs and Orchestration
A beginner-friendly introduction to LLMs, why we need LangChain, and how LangGraph helps build workflow-based AI systems.
Part 7 - Observability with LangSmith: Debug and Monitor Your AI
Learn how to use LangSmith to debug, trace, and monitor your AI applications - find issues fast and optimize performance.
Part 6 - Project: Build Your Own AI Agent
Step-by-step guide to building a complete agentic application with tools, memory, and workflows - a hands-on project for beginners.
Part 4 - RAG Basics: Teaching AI About Your Documents
Learn how to build RAG (Retrieval Augmented Generation) systems that let AI answer questions from your documents - explained simply.
Part 3 - Tool Calling: Giving Your AI Real-World Superpowers
Learn how to extend LLMs with real-world capabilities - from calling APIs to using MCP adapters - explained in simple terms.
Part 5 - LangGraph & Agents: Building Intelligent Workflows
Learn how to build sophisticated agentic systems with LangGraph using states, nodes, edges, and workflows - explained simply.
Case Study: Amazon Rufus - AI Shopping Assistant Done Right
How Amazon's Rufus demonstrates the right way to build AI assistants - agents orchestrate, pipelines execute, policies enforce.
Case Study: How Netflix Solved the 100 Petabyte Problem with Apache Iceberg
A simple story about why Netflix had to invent a new way to manage data, what was breaking, and how the fix changed the entire data industry.
LTAP, the IRC Catalog, and the Hidden Layers of Apache Iceberg
A plain-English guide to what LTAP (Lake Transactional/Analytical Processing) actually means — including the important nuance that it does not replace the transactional database, but instead unifies OLTP and OLAP at the storage layer. Also covers how the Iceberg REST Catalog (IRC) works, and how a SQL query travels through the Iceberg stack from catalog lookup to Parquet rows.
Case Study: How Meta Built Privacy-Aware Data Lineage at Scale (Part 1 - The Challenge)
How Meta discovered that protecting billions of users' privacy at scale requires first answering one deceptively simple question - where does all the data actually go?
Case Study: How Meta Built Privacy-Aware Data Lineage at Scale (Part 2 - The Solution)
How Meta built an automated system that traces billions of data flows across millions of assets in real-time — making privacy enforcement finally possible at scale.
Building a Modern Containerized Open Data Stack
A practical architecture guide for hybrid data platforms in the AI era — separating storage and compute, embracing open table formats, and building portable governed ecosystems.
NumPy Cheat Sheet - Print Reference
Print-friendly NumPy quick reference guide for fast lookup
ORC Deep Dive — Optimized Row Columnar Format Explained
A deep dive into how ORC (Optimized Row Columnar) stores, compresses, and indexes data for Hadoop and Hive workloads.
Pandas Series Cheat Sheet - Quick Reference
Print-friendly quick reference for Pandas Series - creation, indexing, operations, and essential methods.
Parquet Deep Dive — How It Actually Stores Data
Understanding how Parquet stores, encodes, and compresses data under the hood — from row groups to encoding schemes.
Scaling Your API: Part 1 - Performance & Infrastructure
A beginner-friendly guide to scaling your API from handling 1 request per second to 1 million, with diagrams and simple explanations.
Scaling Your API: Part 2 - Design & Architecture
Simple strategies for organizing and designing APIs as your company grows, explained in plain language for everyone.
Scaling Your API Part 4: Load Balancing & High Availability
Learn how to keep your API running smoothly with load balancing and high availability — explained in simple terms anyone can understand.
Scaling Your API Part 5: Monitoring & Performance
Learn how to monitor your API and improve performance — with simple explanations and practical examples anyone can understand.
Statistics Formulas Cheat Sheet
Complete reference guide with all statistical formulas, explanations, and when to use them - your go-to resource for hypothesis testing, confidence intervals, and more.
Case Study: How Uber Built Their Data Infrastructure (Part 1)
How Uber built a data platform to handle 137 million monthly users and billions of events per day — explained simply.
Case Study: How Uber Modernized Their Data Infrastructure with GCP (Part 2)
How Uber migrated from managing thousands of servers to Google Cloud Platform — reducing complexity by 50% and costs by 40%.
Unified Data Infrastructure Layer (EDW + Snowflake + Databricks)
A very simple, practical reference architecture for a unified data platform across on-prem EDW + Snowflake + Databricks — implemented incrementally.
Windsurf IDE Cheat Sheet
Print-friendly quick reference for Windsurf IDE - keyboard shortcuts, Cascade commands, and essential features.