Your First Machine Learning Model, End to End

There is a moment in every data science journey where you stop reading about machine learning and train an actual model. This guide is that moment. We'll go from a raw dataset to an honest performance number using scikit-learn, and — more importantly — we'll do it in the right order, because the order is where beginners get silently burned.

The task

We'll predict whether a passenger survived the Titanic — the classic beginner dataset, because it's small, real, and full of authentic mess: missing values, mixed types, and columns that look useful but aren't.

import pandas as pd

df = pd.read_csv("titanic.csv")
df.shape          # (891, 12)
df["Survived"].value_counts(normalize=True)
# 0    0.616
# 1    0.384

That last line matters more than it looks: 61.6% of passengers died. A "model" that predicts death for everyone scores 61.6% accuracy while learning nothing. That number — the majority-class baseline — is the score to beat. Write it down before you train anything.

Split first. Always first.

The most common beginner mistake is exploring, cleaning, and engineering features on the entire dataset, then splitting at the end. The problem: information from your test rows (their means, their categories, their quirks) has already leaked into your decisions. Your final score will look better than the model actually is — and you won't know it.

from sklearn.model_selection import train_test_split

X = df.drop(columns=["Survived"])
y = df["Survived"]

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42, stratify=y
)

stratify=y keeps the survival ratio identical in both halves. random_state makes the split reproducible. The test set now goes in a drawer — we touch it exactly once, at the very end.

Features: less is more (at first)

Titanic has 12 columns. A first model should use the few with obvious signal:

• Sex — "women and children first" is visible in the data.
• Pclass — ticket class is a proxy for cabin location and wealth.
• Age — with ~20% missing values to handle.
• Fare — skewed, but informative.

We need two preprocessing steps: fill missing ages, and turn text categories into numbers. Both must be fit on training data only — the imputation value comes from the training rows, then gets applied to test rows. A Pipeline makes that automatic:

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder

numeric = ["Age", "Fare"]
categorical = ["Sex", "Pclass"]

prep = ColumnTransformer([
    ("num", SimpleImputer(strategy="median"), numeric),
    ("cat", OneHotEncoder(handle_unknown="ignore"), categorical),
])

This object is the professional habit worth learning early: preprocessing that lives inside the model, so it can never be accidentally fit on data it shouldn't see.

Train two models

Always train a simple model and a stronger one — the gap between them tells you where you are.

from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import cross_val_score

logreg = Pipeline([("prep", prep), ("model", LogisticRegression(max_iter=1000))])
forest = Pipeline([("prep", prep), ("model", RandomForestClassifier(n_estimators=300, random_state=42))])

cross_val_score(logreg, X_train, y_train, cv=5).mean()  # ≈ 0.79
cross_val_score(forest, X_train, y_train, cv=5).mean()  # ≈ 0.81

Note we're scoring with cross-validation on the training set — five different train/validation splits, averaged — not with the test set. The test set is still in its drawer.

Both models clear the 0.616 baseline by a wide margin: the features carry real signal. The forest edges out logistic regression, but not dramatically — on small tabular datasets, that's typical.

The one look at the test set

forest.fit(X_train, y_train)
forest.score(X_test, y_test)   # ≈ 0.80

Around 80%, consistent with cross-validation — which is exactly what we want to see. If the test score had come in far below the CV score, that would smell of overfitting to validation choices; far above, of luck or leakage.

Accuracy isn't the whole story. A confusion matrix shows which mistakes the model makes — and on imbalanced problems you'd reach for precision, recall, and PR-AUC instead. But that's the next lesson, not the first one.

What made this "honest"

Two habits did all the work:

1. Split before you look. Every decision — imputation values, encodings, features — was made using training data only.
2. Beat the baseline, not just zero. 80% only means something because we knew 61.6% was free.

Models are easy; discipline is the skill. If you want to run this whole workflow interactively — data loading, pipelines, cross-validation, live in your browser with nothing to install — the Machine Learning module walks through each piece with editable code.