ML Quick Reference - Libraries, Metrics and Cheat Sheet

A single page you can come back to whenever you need to remember which library, which class, or which metric to use. Everything here is covered in detail in the earlier tutorials. This page is the pocket reference.

Core Libraries

The handful of libraries that show up in every ML project.

Install command for the whole stack:

pip install numpy pandas scikit-learn matplotlib seaborn scipy xgboost lightgbm catboost statsmodels

Algorithms - which class to import

Every model in this handbook, with its scikit-learn (or other) import.

Regression (predict a number)

Classification (predict yes/no or a category)

Clustering (find groups in unlabelled data)

Preprocessing - the tools you will use every time

Evaluation Metrics

Regression metrics

For regression problems (predicting a number). All available in sklearn.metrics.

from sklearn.metrics import (
    mean_absolute_error,    # MAE
    mean_squared_error,     # MSE (and RMSE with squared=False)
    r2_score,
    mean_absolute_percentage_error,
)

mae  = mean_absolute_error(y_true, y_pred)
mse  = mean_squared_error(y_true, y_pred)
rmse = mean_squared_error(y_true, y_pred, squared=False)
r2   = r2_score(y_true, y_pred)

Rule of thumb. Report RMSE and R² together. RMSE tells you how big the typical error is. R² tells you how much of the variance you captured.

Classification metrics

For classification problems (yes/no or a category).

The confusion matrix

Every classification metric is built from these four numbers.

Key metrics

from sklearn.metrics import (
    accuracy_score,
    precision_score,
    recall_score,
    f1_score,
    roc_auc_score,
    log_loss,
    confusion_matrix,
    classification_report,
)

print(classification_report(y_true, y_pred))   # all in one
print(confusion_matrix(y_true, y_pred))
auc = roc_auc_score(y_true, y_pred_proba)      # NOTE: needs probabilities, not labels

Rule of thumb.

• Balanced classes → report Accuracy and F1.
• Imbalanced classes → ignore accuracy, report Precision, Recall, F1, ROC-AUC, PR-AUC.
• Probabilities matter → report Log Loss and ROC-AUC.

Clustering metrics

There are no ground-truth labels in clustering, so we measure cluster quality differently.

from sklearn.metrics import (
    silhouette_score,
    davies_bouldin_score,
    calinski_harabasz_score,
    adjusted_rand_score,
)

sil = silhouette_score(X, labels)
db  = davies_bouldin_score(X, labels)
ch  = calinski_harabasz_score(X, labels)

Distance and Similarity Metrics

The metrics behind clustering, KNN, and similarity search. All in scipy.spatial.distance or sklearn.metrics.pairwise.

Choose the Right Metric for the Job

A condensed cheat sheet to point you to the right metric immediately.

Choose the Right Algorithm for the Job

The condensed pick-an-algorithm cheat sheet.

Hyperparameters You Will Tune Most Often

The knobs that actually move the needle for each algorithm.

Generic tuning template

from sklearn.model_selection import GridSearchCV

param_grid = {
    "n_estimators": [100, 200, 500],
    "max_depth":    [3, 5, 7, None],
}
grid = GridSearchCV(
    estimator=YourModel(),
    param_grid=param_grid,
    cv=5,
    scoring="roc_auc",     # or "f1", "accuracy", "neg_root_mean_squared_error"
    n_jobs=-1,
    verbose=1,
)
grid.fit(X_train, y_train)
print(grid.best_params_, grid.best_score_)

A Reusable Workflow Template

Drop this into any new ML project. It covers the 90% case.

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.metrics import classification_report, roc_auc_score

# 1. Load
df = pd.read_csv("data.csv")
X = df.drop(columns=["target"])
y = df["target"]

# 2. Split
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

# 3. Pipeline (scale + model)
from sklearn.ensemble import RandomForestClassifier
pipe = Pipeline([
    ("scaler", StandardScaler()),
    ("model",  RandomForestClassifier(random_state=42)),
])

# 4. Tune
param_grid = {
    "model__n_estimators": [100, 300, 500],
    "model__max_depth":    [None, 5, 10],
}
grid = GridSearchCV(pipe, param_grid, cv=5, scoring="roc_auc", n_jobs=-1)
grid.fit(X_train, y_train)

# 5. Evaluate on held-out test set
best_model = grid.best_estimator_
y_pred  = best_model.predict(X_test)
y_proba = best_model.predict_proba(X_test)[:, 1]
print("Best params:", grid.best_params_)
print("Test ROC-AUC:", roc_auc_score(y_test, y_proba))
print(classification_report(y_test, y_pred))

For regression, swap RandomForestClassifier → RandomForestRegressor, stratify=y → stratify=None, and use scoring="neg_root_mean_squared_error".

Quick Symbol Reference

Notation that shows up in every ML tutorial.

One Last Tip

If you forget everything else on this page, remember three things.

1. Always scale your features before using any distance-based algorithm (KNN, K-Means, SVM, regularized linear models).
2. Never evaluate on training data. The test score is the only score that matters.
3. Start with the simplest model that could possibly work. If a linear model already does well, you are probably done. Only reach for boosting and ensembles when you actually need the extra accuracy.

That is the entire classical ML toolkit in one page. Bookmark this and come back to it any time you forget which class to import or which metric to use.