Quick tech ticks and comparisons
Hi! I'm Rilov Paloly Kulankara, and this is where I share everything I'm learning in the world of technology. As a software engineering leader, I'm constantly exploring new concepts in data engineering, cloud architecture, security, and scalability. This handbook is my way of documenting these learnings in simple, beginner-friendly guides with diagrams and real-world examples. Whether you're a seasoned engineer or just starting out, I hope you find these notes helpful on your own learning journey!
A simple next step after the beginner CNN guide. Explains advanced CNN ideas in plain English, with diagrams throughout — deeper networks and the degradation problem, overfitting, dropout, batch normalization, residual connections, global average pooling, 1x1 bottleneck convolutions, CNN backpropagation through filters, feature maps, ReLU, and pooling, transfer learning, data augmentation, and modern CNN architecture patterns.
A beginner-friendly guide to anomaly detection. Learn how to find unusual data points using isolation forest, one-class SVM, and statistical methods. Covers fraud detection, network intrusion, and quality control with full Python examples.
A beginner-friendly guide to association rules and the Apriori algorithm. Learn how to find "customers who bought X also bought Y" patterns in transaction data, with support, confidence, lift explained and full Python examples.
A complete hands-on walkthrough of building, training, and evaluating a CNN on the CIFAR-10 dataset using PyTorch. Covers loading and preprocessing the dataset, defining a CNN architecture, writing the training loop with validation, and evaluating with accuracy, confusion matrix, and classification report.
An overview of the main real-world applications of CNNs — image classification, object detection, image segmentation, facial recognition and analysis, and optical character recognition (OCR). Explains what each task is, how CNNs are used, key architectures, and real-world examples.
Covers the full CNN model design pipeline — from data collection and preprocessing through training — then explains when to train from scratch versus use transfer learning or fine-tuning, and finally shows how to visualize what a CNN has actually learned using Grad-CAM and activation maps.
Learn the fundamentals of Exploratory Data Analysis (EDA) - understanding your data through visualization, summary statistics, and pattern discovery.
Learn how to make predictions about populations from samples, understand confidence intervals, and test hypotheses using statistical methods.
Master the fundamentals of hypothesis testing - learn the 5-step process, understand p-values, and avoid common errors in statistical testing.
Comprehensive guide to statistical hypothesis testing methods - single sample tests, two sample tests, and proportion tests with practical examples.
Learn single sample hypothesis tests - z-tests and t-tests for comparing one sample to a claimed value, with practical examples and Python code.
Master two sample hypothesis tests - independent and paired t-tests for comparing two groups, with real-world examples and Python implementations.
Learn proportion tests for testing percentages and proportions - one-sample and two-sample proportion tests with practical examples and Python code.
A beginner-friendly guide to classical text generation. Learn how computers generate language using N-gram models, Markov chains, probability, smoothing techniques, and how to evaluate quality with perplexity—before the era of neural networks.
Learn how classical NLP systems generate sentences, automatically summarise documents, and translate between languages—before neural networks. Covers controlled sentence generation, extractive summarisation, and rule-based and statistical machine translation.
Learn how deep learning mirrors ideas from the brain, using the spam-detection example throughout. Covers pattern recognition, memory, attention, generalisation, and learning from mistakes.
A zero-background introduction to Convolutional Neural Networks. Starts with a simple cat-vs-dog photo example and builds up, one idea at a time, through convolution, activation, feature maps, padding, and pooling — then goes further into the exact output-size and parameter-count formulas, finishing with a full layer-by-layer walkthrough of the real VGG16 architecture.
Learn how to load, batch, shuffle, and iterate over spam-detection data in PyTorch using torch.utils.data.Dataset and DataLoader. Uses the same email example throughout.
A simple beginner friendly guide to decision trees. Part 1 covers what they are, why we use them, and how they actually work step by step using real life examples.
Part 2 of the decision trees tutorial. We explain in very simple terms what impurity means, how to measure it using Gini and entropy, and how the algorithm decides which question to ask using information gain.
Part 3 of the decision trees tutorial. We walk through the full training process, talk about overfitting and pruning, discuss real world advantages and disadvantages, and then build a working decision tree in Python on a real dataset.
A one-page reference for every deep learning formula and concept from the tutorial series, all using the spam-detection example. Each formula has a memory trick.
A simple introduction to ensemble learning. Why combining many weak models beats relying on one strong model. The wisdom of crowds, bias and variance, and the two big families of ensembles.
Part 2 of the ensembles tutorial. We explain bootstrap sampling, build up to the Random Forest algorithm step by step, and walk through full hyperparameter tuning using number of trees, maximum depth, and grid search with cross validation.
Part 3 of the ensembles tutorial. We explain boosting from the ground up, walk through AdaBoost and Gradient Boosting in plain language, meet the modern champions like XGBoost and LightGBM, and finish with practical advice on when to use each method.
Learn exactly what happens during forward and backward propagation, using a two-layer spam-detection model. Worked numerical example with the chain rule included.
The direct continuation of Part 1. Part 1 built the spam model with handcoded weights. Part 2 answers the natural next question — how does a model actually learn those weights? Covers labels, loss, backpropagation, and the complete training loop using the same spam example.
A beginner-friendly guide to Gaussian Mixture Models (GMMs). Learn how probabilistic clustering differs from K-Means, what the EM algorithm does, and how to fit GMMs in Python for soft cluster assignments.
A complete beginner friendly guide to hierarchical clustering. We cover the agglomerative algorithm step by step, the different linkage methods, how to read a dendrogram, practical considerations, and a full Python demonstration.
A complete beginner friendly guide to K-Means Clustering. We cover unsupervised learning, how K-Means actually works step by step, picking the right number of clusters, common pitfalls, and a full Python demonstration.
A complete beginner friendly guide to K-Nearest Neighbours (KNN). We cover how it works, weighted aggregation, picking K, key practical considerations, and a full Python demonstration for both classification and regression.
A beginner-friendly guide to LSTM and GRU — the improved versions of RNN that can remember things from much longer ago. Learn why basic RNNs forget, how LSTM gates solve this, and how to use both in PyTorch.
A beginner-friendly guide to Lexical Processing in NLP. Learn how computers break down text through tokenization, text normalization, stopword removal, stemming, lemmatization, and spell correction—with simple explanations and Python examples.
Deep dive into Linear Regression assumptions, challenges, and advanced techniques - scaling, feature engineering, handling violations, and regularization explained simply.
Master Linear Regression with expert-level topics - categorical encoding, deep multicollinearity, influential points, cross-validation, polynomial features, and real-world techniques.
Complete beginner's guide to Linear Regression - learn to predict continuous values with real-world examples and Python code.
Master Logistic Regression from zero to hero - the most popular algorithm for classification. Beginner-friendly explanations with real examples and Python code.
One flashcard per algorithm. The core idea, an everyday analogy, when to use it, and a tiny code snippet. Perfect for review, revision, or refreshing your memory right before an interview.
A symptom to cause to fix troubleshooting guide for the most common machine learning problems. When your model behaves weirdly, look up the symptom here and find out what is going wrong.
A complete end-to-end ML project on the Titanic dataset, from raw CSV to final evaluation. We do data exploration, cleaning, feature engineering, model comparison, hyperparameter tuning, and final test set evaluation - showing exactly how all the pieces from the previous tutorials fit together.
A frequently asked questions page answering every common 'what's the difference between X and Y' question in machine learning. Quick, clear, side-by-side comparisons of the concepts that confuse beginners most.
An A-to-Z plain English dictionary of every machine learning term used in this handbook. No equations, no jargon defined with more jargon. Just clear short explanations for every concept you will meet.
A single-page quick reference for every ML topic in this handbook. Libraries to import, classes to use, key hyperparameters, evaluation metrics with formulas, and a master cheat sheet for picking the right tool and the right metric.
A side-by-side visual tour of how every classifier carves up the same 2D dataset. See exactly why linear models draw straight lines, decision trees make boxes, KNN makes wavy regions, and ensembles smooth things out.
A friendly story-style tour of every machine learning technique covered in this handbook. No math, no code, no jargon. Just a clear, plain English explanation of what each tool does and when you would actually use it in real life.
A friendly, story-style introduction to Machine Learning. What it is, how it differs from regular programming, the three big types, the full ML pipeline, and the key ideas every beginner should know. With diagrams.
A beginner-friendly guide to how machine learning works with text. Learn how computers understand language through simple, real-world examples—no heavy math, just clear explanations of sentiment analysis, spam detection, named entity recognition, and more.
Learn probability from zero using simple questions, formulas, plain-English explanations, and worked answers.
Count possible outcomes using the multiplication rule, permutations, and combinations before calculating probability.
Learn conditional probability using simple class examples, complete formulas, input validation, and a Python solution.
Learn whether events affect each other and use Bayes' theorem to update probability after new evidence.
Understand random variables, probability distributions, expected value, variance, and standard deviation through simple questions.
Connect probability ideas using joint distributions, covariance, conditional expectation, the law of large numbers, and the central limit theorem.
A quick-reference cheat sheet covering every important NLP term and concept from the full tutorial series. Organised by topic—from tokenisation to machine translation—with plain-English definitions and one-line memory aids.
A complete reference guide to every Python library used across the NLP tutorial series. Learn what each library does, how to install it, and which functions to use for each NLP task—with ready-to-run code examples.
A complete beginner-friendly guide to Naive Bayes. Learn how probability drives classification, why the "naive" assumption works in practice, and build a spam detector step by step in Python.
A beginner-friendly introduction to Natural Language Processing. Learn what NLP is, where it's used across industries, the stages of the NLP pipeline, and the different approaches to building NLP systems—using simple stories and real-world examples.
A complete beginner-friendly guide to PCA and dimensionality reduction. Learn why too many features hurt models, how PCA finds the directions of maximum variance, and how to apply it in Python step by step.
Learn how PyTorch creates, tracks, and updates learnable parameters for a spam-detection model. Understand weights, biases, requires_grad, and how to inspect and manage model parameters.
A beginner-friendly introduction to Recurrent Neural Networks. Learn why normal neural networks cannot handle sequences, how an RNN reads data step by step, and how to build one in PyTorch using simple everyday examples.
A beginner-friendly guide to practical RNN applications. Learn how to use LSTM for time series forecasting, sequence classification, and sequence labelling with clear step-by-step Python examples.
Master Regularization from zero - Ridge, Lasso, Elastic Net explained simply with analogies, math, and Python code. Stop overfitting forever!
A beginner-friendly guide to reinforcement learning. Learn how an agent learns by trial and error using rewards and punishments, how Q-learning works, and how RLHF is used to train language models like ChatGPT.
A beginner-friendly guide to self-supervised learning — how models train themselves on unlabelled data by predicting parts of the input from other parts. Covers masked prediction, contrastive learning, and the role of SSL in GPT and BERT.
A beginner-friendly guide to how computers understand word meaning. Covers lexical semantics, word sense disambiguation, co-occurrence models, and how to measure word similarity using statistics.
Learn how computers identify who is doing what to whom in a sentence. Covers Semantic Role Labelling, Named Entity Recognition, IOB tagging, Conditional Random Fields, and Coreference Resolution—with simple explanations and Python examples.
A beginner-friendly guide to semi-supervised learning — the technique that uses a small amount of labelled data and a large amount of unlabelled data together. Covers self-training, label propagation, and GANs with Python examples.
A complete beginner friendly guide to the metrics machine learning uses to decide whether two points are similar or different. We cover distance-based, angle-based, correlation-based, and set-based metrics with simple examples and Python code.
A complete beginner-friendly guide to Support Vector Machines. Learn how SVMs find the best boundary between classes, what the kernel trick does, and how to use SVMs in Python for real classification tasks.
Learn how computers identify word types (nouns, verbs, adjectives) and group them into meaningful phrases. A beginner-friendly guide to Part-of-Speech tagging and shallow parsing with Python examples.
Learn how computers understand sentence structure through grammar rules, constituency parsing, and dependency parsing. A beginner-friendly guide to analyzing how words relate to each other.
Learn how to visualize sentence structure with parse trees and ensure grammatical correctness through agreement checking. The final part of our syntactic processing series.
Learn tensors and tensor operations using the same spam-detection example. Covers feature vectors, batches of emails, weights, matrix multiplication, broadcasting, and reshaping. Builds directly on the training loop from Part 2.
Learn how computers turn raw text into numbers, and how to discover hidden topics in large collections of documents—without any labelled data. Covers Bag-of-Words, TF-IDF, Word Embeddings, and Topic Modelling with NMF.
Learn how a neural network decides whether an email is spam by turning real-world information into numbers, multiplying by weights, adding a bias, and applying an activation function. Uses a single, concrete example throughout.
Why Agentic AI isn't failing because it's weak it's failing because we're asking it to do the wrong job. Agents should orchestrate workflows, not replace them.
How Airbnb discovered their biggest data problem wasn't big data — it was getting everyone to agree on what "bookings" actually means.
How Airbnb built Minerva — a platform that turned 10,000 inconsistent metrics into a single source of truth, saving 60% of engineering time.
A user-friendly technical guide to Apache Iceberg's architecture - how it works under the hood, why it's different from traditional tables, and why it's the go-to choice for AI/ML workloads.
How Apache Avro enables schema evolution, efficient serialization, and seamless streaming data with embedded schemas.
Learn how to choose the right database for your API — from handling thousands to millions of requests per second.
A beginner-friendly guide to every data file format from CSV to Delta Lake with simple explanations and visual examples.
Understand different database types (RDBMS, Columnar, NoSQL, Vector) with simple explanations, popular options, and when to use each.
How Delta Lake brings ACID transactions, time travel, and schema evolution to Parquet files on data lakes.
Learn the fundamentals of Kimball dimensional modeling with simple analogies, diagrams, and practical examples.
A beginner-friendly guide comparing hashing and encryption with simple analogies and Python code examples.
A deep dive into how query engines like Trino and Spark execute queries against Iceberg tables — from REST Catalog API calls to reading Parquet files from S3.
Learn how to run Apache Spark on Kubernetes using the Spark Operator. This guide covers the basics: what it is, how it works, and how to submit your first Spark job without manual spark-submit commands.
Enterprise-grade Spark on Kubernetes: multi-tenant architecture, centralized logging and monitoring, self-service job submission portals, cost tracking, and security best practices for running Spark at scale.
Learn the core LangChain building blocks - chat models, prompting patterns, structured outputs, and chaining - explained simply.
A beginner-friendly introduction to LLMs, why we need LangChain, and how LangGraph helps build workflow-based AI systems.
Learn how to use LangSmith to debug, trace, and monitor your AI applications - find issues fast and optimize performance.
Step-by-step guide to building a complete agentic application with tools, memory, and workflows - a hands-on project for beginners.
Learn how to build RAG (Retrieval Augmented Generation) systems that let AI answer questions from your documents - explained simply.
Learn how to extend LLMs with real-world capabilities - from calling APIs to using MCP adapters - explained in simple terms.
Learn how to build sophisticated agentic systems with LangGraph using states, nodes, edges, and workflows - explained simply.
A simple story about why Netflix had to invent a new way to manage data, what was breaking, and how the fix changed the entire data industry.
A plain-English guide to what LTAP (Lake Transactional/Analytical Processing) actually means — including the important nuance that it does not replace the transactional database, but instead unifies OLTP and OLAP at the storage layer. Also covers how the Iceberg REST Catalog (IRC) works, and how a SQL query travels through the Iceberg stack from catalog lookup to Parquet rows.
How Meta discovered that protecting billions of users' privacy at scale requires first answering one deceptively simple question - where does all the data actually go?
How Meta built an automated system that traces billions of data flows across millions of assets in real-time — making privacy enforcement finally possible at scale.
A practical architecture guide for hybrid data platforms in the AI era — separating storage and compute, embracing open table formats, and building portable governed ecosystems.
Print-friendly NumPy quick reference guide for fast lookup
A deep dive into how ORC (Optimized Row Columnar) stores, compresses, and indexes data for Hadoop and Hive workloads.
Print-friendly quick reference for Pandas Series - creation, indexing, operations, and essential methods.
Understanding how Parquet stores, encodes, and compresses data under the hood — from row groups to encoding schemes.
How Amazon's Rufus demonstrates the right way to build AI assistants - agents orchestrate, pipelines execute, policies enforce.
A beginner-friendly guide to scaling your API from handling 1 request per second to 1 million, with diagrams and simple explanations.
Simple strategies for organizing and designing APIs as your company grows, explained in plain language for everyone.
Learn how to keep your API running smoothly with load balancing and high availability — explained in simple terms anyone can understand.
Learn how to monitor your API and improve performance — with simple explanations and practical examples anyone can understand.
Complete reference guide with all statistical formulas, explanations, and when to use them - your go-to resource for hypothesis testing, confidence intervals, and more.
How Uber built a data platform to handle 137 million monthly users and billions of events per day — explained simply.
How Uber migrated from managing thousands of servers to Google Cloud Platform — reducing complexity by 50% and costs by 40%.
A very simple, practical reference architecture for a unified data platform across on-prem EDW + Snowflake + Databricks — implemented incrementally.
Print-friendly quick reference for Windsurf IDE - keyboard shortcuts, Cascade commands, and essential features.