Probability Basics
Probability is the mathematics of chance. It gives you a number between 0 and 1 for how likely something is, and it underpins the statistics you use to decide whether a pattern in data is real or just luck.
What a probability means
A probability of 0 means impossible, 1 means certain, and 0.5 means as likely as not. When all outcomes are equally likely:
P(event) = number of outcomes in the event ÷ total number of outcomes
For a fair six-sided die, P(rolling a 6) = 1/6 ≈ 0.167. With two dice, Python can list every outcome and count for you:
from fractions import Fraction outcomes = [(a, b) for a in range(1, 7) for b in range(1, 7)] sevens = [pair for pair in outcomes if sum(pair) == 7] print(len(outcomes), len(sevens)) print(Fraction(len(sevens), len(outcomes))) print(round(len(sevens) / len(outcomes), 4))
There are 36 equally likely pairs and 6 of them add up to 7, so P(total of 7) = 6/36 = 1/6. Seven is the most likely total with two dice.
The basic rules
| Rule | Formula | Example |
|---|---|---|
| Complement ("not") | P(not A) = 1 − P(A) | P(not a 6) = 1 − 1/6 = 5/6 |
| "Or" (addition) | P(A or B) = P(A) + P(B) − P(A and B) | P(even or above 4) = 3/6 + 2/6 − 1/6 = 4/6 (the 6 is in both, so subtract it once) |
| "And" for independent events | P(A and B) = P(A) × P(B) | Two coin flips both heads = 1/2 × 1/2 = 1/4 |
| Conditional | P(A | B) = P(A and B) ÷ P(B) | The chance of A given that B happened |
Events are independent when one doesn't change the chance of the other, like two separate coin flips. Many real events are not independent: rain today makes rain tomorrow more likely.
Probability from data
In data science you usually estimate probabilities from how often things happened: the relative frequency. Here are ten orders from a shop, with the payment method and whether the order was above KES 1,000:
import pandas as pd
customers = pd.DataFrame({
"payment": ["M-Pesa", "Cash", "M-Pesa", "M-Pesa", "Cash",
"M-Pesa", "Cash", "M-Pesa", "M-Pesa", "Cash"],
"big_order": [True, False, True, False, False,
True, True, False, True, False],
})
print(customers["big_order"].mean()) # P(big order)
print(customers.groupby("payment")["big_order"].mean().round(3)) # P(big | payment)
The mean of a True/False column is the share of True values, so P(big order) = 5/10 = 0.5. Grouping gives conditional probabilities: P(big | M-Pesa) = 4/6 ≈ 0.667 and P(big | Cash) = 1/4 = 0.25. (pd.crosstab(customers["payment"], customers["big_order"]) shows the underlying counts.) Ten orders is a tiny sample, so treat these as rough estimates.
P(A | B) is not the same as P(B | A). P(big | M-Pesa) = 4/6 asks "of the M-Pesa orders, what share were big?", while P(M-Pesa | big) = 4/5 asks "of the big orders, what share were paid by M-Pesa?". Different questions give different answers. In medicine the difference is crucial: P(positive test | disease) can be high while P(disease | positive test) is low if the disease is rare.
Simulation and the law of large numbers
You can estimate a probability by simulating an experiment many times. The law of large numbers says that the proportion of times an event happens gets closer to its true probability as the number of trials grows.
import numpy as np
rng = np.random.default_rng() # a random number generator
rolls = rng.integers(1, 7, size=10_000) # 1 to 6 (the 7 is excluded)
for n in [10, 100, 10_000]:
share = (rolls[:n] == 6).mean()
print(f"After {n} rolls, share of sixes: {share}")
Your numbers will differ because the rolls are random, but the last line will be close to 1/6 ≈ 0.1667, while the first can be far off. To get the same numbers on every run, give the generator a seed: np.random.default_rng(42).
The law of large numbers is about the long run. It does not mean a six is "due" after several rolls without one; each roll is independent. Believing otherwise is the gambler's fallacy. Betting companies set their odds so that, on average and over many bets, they win and bettors lose.
Probability distributions
A distribution describes all the possible values of a random quantity and how likely each is. SciPy's scipy.stats module has the common ones.
Binomial: counting successes
Use the binomial distribution for the number of successes in a fixed number of independent tries, each with the same chance of success.
from scipy.stats import binom # 5 fair coin flips: chance of exactly 3 heads print(round(binom.pmf(3, 5, 0.5), 4)) # A SACCO gives 10 loans; each has a 10% chance of default. # Chance of at most 1 default: print(round(binom.cdf(1, 10, 0.1), 4))
pmf gives the probability of exactly one value (10 of the 32 equally likely flip sequences have 3 heads, and 10/32 = 0.3125). cdf gives the probability of that value or less.
Normal: the bell curve
Many measurements, such as heights or exam scores, roughly follow the symmetrical, bell-shaped normal distribution, described by its mean and standard deviation. About 68% of values lie within 1 standard deviation of the mean, about 95% within 2, and about 99.7% within 3.
from scipy.stats import norm # Suppose heights have mean 165 cm and standard deviation 8 cm print(round(norm.cdf(173, loc=165, scale=8), 4)) # shorter than 173 cm print(round(norm.cdf(173, 165, 8) - norm.cdf(157, 165, 8), 4)) # within 1 sd print(round(1 - norm.cdf(181, 165, 8), 4)) # taller than 181 cm
181 cm is 2 standard deviations above the mean, and only about 2.3% of people are taller than that under this model.
| Distribution | Describes | Example |
|---|---|---|
| Uniform | All values equally likely | One roll of a fair die |
| Binomial | Number of successes in n tries | Loan defaults out of 10 loans |
| Poisson | Number of events in a fixed time | Customers arriving per hour |
| Normal | Measurements clustering around a mean | Heights, measurement errors |
Key points
- Probabilities run from 0 (impossible) to 1 (certain).
- Use the complement, addition, multiplication and conditional rules, and check independence before multiplying.
- From data, estimate probabilities as relative frequencies; a True/False column's mean is a proportion.
- Distributions such as the binomial and normal model how random quantities behave; the next step is Hypothesis Testing.
Tip: use the ← and → keys to move between pages.
