Skip to content
Kuza Ndoto Academy

Correlation

Beginner → IntermediateChapter 5 · Page 17 of 205 min readUpdated 15 Nov 2026

Correlation measures how strongly two numeric variables move together. It is one number between −1 and +1, and it is a quick way to spot relationships worth investigating, as long as you remember that correlation alone never proves that one thing causes another.

Positive, negative and none

Correlation (r)MeaningExample
Close to +1As one goes up, the other goes upHours studied and exam score
Close to −1As one goes up, the other goes downPrice of a product and units sold
Close to 0No straight-line relationshipShoe size and exam score

A rough guide to strength, using the size of r and ignoring the sign: below 0.2 very weak, 0.2–0.4 weak, 0.4–0.6 moderate, 0.6–0.8 strong, above 0.8 very strong. These cut-offs are only a rule of thumb; what counts as "strong" depends on the field.

r close to +1 r close to 0 r close to −1
Figure 1. Illustration (made-up points): a strong positive relationship, no linear relationship, and a strong negative relationship.

Pearson's correlation coefficient

The most common measure is Pearson's r, which measures the strength of a straight-line relationship:

r = Σ(x − x̄)(y − ȳ) ÷ √[ Σ(x − x̄)² × Σ(y − ȳ)² ]

The top of the fraction is large and positive when x and y are above their means at the same time (and below them at the same time). The bottom scales the answer so it always lands between −1 and +1, whatever the units.

A worked example

A kiosk owner records the daily high temperature and the number of sodas sold on six days. The means are x̄ = 26 °C and ȳ = 140 sodas.

100 120 140 160 180 18 20 22 24 26 28 30 32 34 Daily high temperature (°C) Sodas sold
Figure 2. The six days plotted, with the least-squares line (dashed). The points sit close to the line, which is why r is so high; the 25 °C day falls a little below it.
Temp xSodas yx − x̄y − ȳ(x − x̄)(y − ȳ)(x − x̄)²(y − ȳ)²
20110−6−3018036900
22125−4−156016225
25120−1−20201400
27150110101100
301604208016400
32175635210361225
Totals5601063250

r = 560 ÷ √(106 × 3250) = 560 ÷ √344,500 ≈ 560 ÷ 586.94 ≈ 0.954, a very strong positive correlation. pandas agrees:

import pandas as pd

weather = pd.DataFrame({
    "temp_c": [20, 22, 25, 27, 30, 32],
    "sodas_sold": [110, 125, 120, 150, 160, 175],
})

print(round(weather["temp_c"].corr(weather["sodas_sold"]), 3))
print(weather.corr())
0.954 temp_c sodas_sold temp_c 1.000000 0.954099 sodas_sold 0.954099 1.000000

df.corr() returns a correlation matrix: every column against every other. The diagonal is always 1 (each column with itself), and the matrix is symmetrical. With many columns, show it as a seaborn heatmap. In pandas 2.x, add numeric_only=True if the DataFrame also contains text columns.

Is the correlation real? pearsonr

With only six days, could a correlation this strong appear by luck? scipy.stats.pearsonr returns r together with a p-value.

from scipy.stats import pearsonr

r, p = pearsonr(weather["temp_c"], weather["sodas_sold"])
print(f"r = {r:.3f}, p = {p:.4f}")
r = 0.954, p = 0.0031

The p-value answers: if there were really no linear relationship between temperature and soda sales, how often would six random days produce a correlation at least this strong (in either direction)? The answer is about 0.3% of the time, which is rare, so the relationship is unlikely to be pure chance. It does not tell you the relationship is causal or how big it is in the wider world; with six points the true correlation could be noticeably weaker. The p-value also assumes the data is roughly normally distributed. Hypothesis Testing explains p-values in detail.

Spearman's rank correlation

Spearman's ρ (rho) ranks each variable from smallest to largest and correlates the ranks. It measures whether the relationship is consistently upward or downward (monotonic), even if it isn't a straight line, and it is much less affected by outliers.

print(round(weather["temp_c"].corr(weather["sodas_sold"], method="spearman"), 3))
0.943

Temperatures rank 1 to 6 in order. Soda sales rank 1, 3, 2, 4, 5, 6, because 120 on the 25 °C day is lower than 125 on the 22 °C day. That single swap lowers ρ slightly from a perfect 1. scipy.stats.spearmanr gives the same value with a p-value.

Pitfalls

Correlation is not causation

Umbrella sales and matatu delays in Nairobi rise and fall together, but umbrellas don't cause traffic. Both are driven by a third factor, rain, called a confounder. Even when two things really are connected, correlation can't tell you which causes which. To show cause and effect you need a controlled experiment or careful causal analysis.

Non-linear relationships

Pearson's r only detects straight-line patterns. A perfect U-shape can score zero:

x = pd.Series([-2, -1, 0, 1, 2])
y = x ** 2              # 4, 1, 0, 1, 4: completely determined by x
print(round(x.corr(y), 3))
0.0

y depends entirely on x, yet r is 0. Always draw a scatter plot before trusting a correlation.

Outliers and small samples

One extreme point can create a strong correlation where there is none, or hide a real one. With few data points, r can swing a lot from sample to sample. Check a scatter plot, try Spearman, and report the sample size.

Common mistake

Reading r as a slope or a percentage. r = 0.5 does not mean "half of the sales are explained by temperature" or "sales rise 0.5 per degree". It only describes how tightly the points cluster around a straight line. For the size of the effect, fit a line with linear regression.

Key points

  • Correlation (r) runs from −1 to +1 and measures how closely two variables follow a straight line.
  • Use Series.corr() or DataFrame.corr(); use scipy.stats.pearsonr for a p-value.
  • Spearman's ρ works on ranks, catches monotonic curves and resists outliers.
  • Correlation is not causation; always plot the data and watch for confounders, outliers and curves.
Up next · StatisticsHypothesis TestingIs a difference real or random?
→

Tip: use the ← and → keys to move between pages.