Feature Scaling

The Problem

Some features have large ranges:

• Age: 0-100
• Income: $0 - $1,000,000
• Temperature: -50 to 50°C

Others have small ranges:

• Rating: 0-5

Distance-based algorithms (KNN, K-Means) and Gradient Descent see large-range features as "more important" just because of scale, not signal!

Solution: Normalize Features

StandardScaler (Standardization)

z = (x - mean) / std_dev

• Centers data at 0, scales to std dev 1
• Result: Normally distributed between -3 and 3
• Use for: Algorithms assuming normal distribution (Linear/Logistic Regression, Neural Networks)

MinMaxScaler (Normalization)

scaled = (x - min) / (max - min)

• Scales to [0, 1]
• Use for: Tree-based models (don't need it, but doesn't hurt), NN activation functions expecting [0,1]

RobustScaler

scaled = (x - median) / IQR

• Uses median and IQR instead of mean/std
• Use for: Data with extreme outliers

When NOT to Scale

✗ Tree-based models (Decision Trees, Random Forest, XGBoost) — They're scale-invariant ✓ Distance-based algorithms (KNN, K-Means) ✓ Gradient Descent (Linear/Logistic Regression, Neural Networks) ✓ PCA, SVM