Features & Labels
Every supervised model learns a link between features (the inputs) and a label (the answer). Choosing and preparing good features is often more important than choosing the algorithm, so it pays to understand them well.
Features, labels and the X/y convention
Think of a dataset as a spreadsheet. Each row is one example (also called a sample or observation). Each column you feed into the model is a feature. The column you want to predict is the label, also called the target.
By convention, the feature table is called X (capital, because it's a 2-D matrix) and the labels are called y (lowercase, a 1-D list). Here is a small dataset of rental flats:
import pandas as pd
df = pd.DataFrame({
"size_sqm": [30, 45, 60, 80],
"bedrooms": [1, 1, 2, 3],
"town": ["Nairobi", "Kisumu", "Nairobi", "Nakuru"],
"rent_kes": [15000, 9000, 25000, 30000],
})
X = df.drop(columns="rent_kes") # features: everything except the label
y = df["rent_kes"] # label
print(X.shape)
print(y.tolist())
X has 4 rows and 3 feature columns. Since the label is a number, this is a regression problem.
Types of features
| Type | Examples | How models use it |
|---|---|---|
| Numerical | Size in sqm, age, monthly income | Directly, often after scaling |
| Categorical (nominal) | Town, mobile network, crop type | Must be encoded as numbers, e.g. one-hot |
| Ordinal | Education level, T-shirt size (S, M, L) | Encoded as ordered numbers (0, 1, 2) |
| Boolean | Has a job, owns a smartphone | As 0 and 1 |
| Text, images, dates | Reviews, photos, sign-up date | Converted into numeric features first |
Encoding categorical features
Models do maths, so they need numbers. You can't just replace Nairobi with 1, Kisumu with 2 and Nakuru with 3, because that tells the model Nakuru is "three times" Nairobi. Instead use one-hot encoding: one 0/1 column per category.
X_encoded = pd.get_dummies(X, columns=["town"], dtype=int) print(X_encoded)
Each flat now has a 1 in exactly one town column. In a full scikit-learn project you would usually use OneHotEncoder inside a pipeline instead, because it remembers the categories it saw during training and can safely handle a new town at prediction time with handle_unknown="ignore".
Feature engineering
Feature engineering means creating better inputs from the raw data using your knowledge of the problem. Some examples:
- From a date of birth, compute age.
- From a transaction timestamp, extract hour of day and day of week (fraud often happens at odd hours).
- From total rent and size, compute rent per square metre.
- From a list of M-Pesa transactions, compute average monthly inflow.
A good feature makes the pattern easier to see. Removing useless or duplicate features also helps: they add noise and slow training.
What makes a good feature?
- Related to the label: size clearly relates to rent; the tenant's favourite colour doesn't.
- Available at prediction time: you must have the value when you actually need a prediction.
- Reliable: measured consistently and not mostly missing.
- Fair and lawful to use: some attributes, or close stand-ins for them, can lead to discrimination (see Bias & Fairness).
Target leakage: including a feature that secretly contains the answer. For example, when predicting whether a loan will default, a column like "number of reminder calls made by the recovery team" is only filled in after a borrower falls behind. The model will look brilliant in testing and fail in real life, because that information doesn't exist when the loan is approved.
Key points
- Features (X) are the inputs; the label (y) is what you predict.
- Categorical features need encoding; one-hot encoding avoids fake ordering.
- Feature engineering turns raw data into more useful inputs.
- Only use features that will be available, honestly, at prediction time.
Tip: use the ← and → keys to move between pages.
