Skip to content
Kuza Ndoto Academy

Feature Scaling

IntermediateChapter 2 · Page 7 of 264 min readUpdated 15 Nov 2026

Feature scaling puts numeric features on a similar range. Without it, a feature measured in large units (income in KES) can drown out one measured in small units (age in years), simply because its numbers are bigger. Many algorithms quietly give worse results on unscaled data.

Why scale?

Imagine predicting loan risk from age (20 to 60) and monthly income (KES 20,000 to 200,000). For a distance-based model like K-Nearest Neighbours, two customers who differ by 30 years of age but KES 100 in income look almost identical, because 100 is bigger than 30. Income dominates purely because of its units. Scaling fixes this.

Usually needs scalingUsually doesn't
KNN, k-means, SVMs, PCA, neural networks, logistic and linear regression with regularisation, anything trained by gradient descentDecision trees, random forests and other tree-based models (they split on one feature at a time, so units don't matter)

Normalisation (min-max scaling)

Min-max scaling squeezes each feature into the range 0 to 1:

x' = (x − min) / (max − min)

For ages 20, 30 and 40: min = 20, max = 40, so 20 becomes 0, 30 becomes (30 − 20) / 20 = 0.5, and 40 becomes 1. It keeps the shape of the data but is sensitive to outliers: one age of 120 would push all normal ages close to 0.

Standardisation (z-score scaling)

Standardisation shifts each feature to have mean 0 and standard deviation 1:

z = (x − mean) / standard deviation

For ages 20, 30 and 40: the mean is 30. The standard deviation is the square root of the average squared distance from the mean: √((100 + 0 + 100) / 3) = √66.67 ≈ 8.165. So 40 becomes (40 − 30) / 8.165 ≈ 1.22, 30 becomes 0 and 20 becomes −1.22. A z-score tells you how many standard deviations a value is from average. This is the default choice for most models.

Scaling with scikit-learn

import numpy as np
from sklearn.preprocessing import MinMaxScaler, StandardScaler

X = np.array([[20, 20000],
              [30, 50000],
              [40, 80000]])      # columns: age, monthly income (KES)

print(MinMaxScaler().fit_transform(X))
print(StandardScaler().fit_transform(X))
[[0. 0. ] [0.5 0.5] [1. 1. ]] [[-1.22474487 -1.22474487] [ 0. 0. ] [ 1.22474487 1.22474487]]

After scaling, age and income are on the same footing, even though income was originally a thousand times larger. (Both columns give identical results here because each is evenly spaced, which makes the arithmetic easy to check.)

Raw values on one axis age income (KES) 020,00040,00060,00080,000 ages 20, 30 and 40 all land here After standardisation (z-scores) -1.5-1-0.500.511.5 -1.2201.22
Figure 1. On the raw scale, income's big numbers hide age completely. After standardisation, both features take the values -1.22, 0 and 1.22, so they count equally.
Note

Other scalers exist: RobustScaler uses the median and IQR, so it copes better with outliers. Scaling applies only to numeric features; one-hot columns are already 0 and 1.

Fit on training data, transform everything

A scaler learns numbers from data: the min and max, or the mean and standard deviation. Those numbers must come from the training set only. You then apply the same transformation to the test set and to any future data.

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)   # learn mean and std from training data
X_test_scaled = scaler.transform(X_test)         # reuse them: no fitting on test data

Even better, put the scaler in a pipeline so this happens automatically, including inside cross-validation:

from sklearn.pipeline import make_pipeline
from sklearn.neighbors import KNeighborsClassifier

model = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=5))
model.fit(X_train, y_train)
Common mistake

Scaling before splitting. If you call fit_transform on the whole dataset and split afterwards, the mean and standard deviation include the test rows. Information about the test set has leaked into training. The effect is often small, but it makes your evaluation dishonest, and with some steps it can be large. Split first, then fit the scaler on the training set.

Common mistake

Calling fit_transform on the test set as well. This rescales the test data using its own statistics, so the same raw value can map to different scaled values in training and testing. Use transform only.

Key points

  • Scaling stops large-unit features from dominating distance- and gradient-based models.
  • Min-max maps to 0–1; standardisation gives mean 0 and standard deviation 1.
  • Tree-based models don't need scaling.
  • Fit the scaler on training data only, ideally inside a pipeline.
Up next · Data PreparationTrain/Test SplitKeep data aside to test honestly.
→

Tip: use the ← and → keys to move between pages.