Skip to content
Kuza Ndoto Academy

Features & Labels

IntermediateChapter 2 · Page 5 of 263 min readUpdated 15 Nov 2026

Every supervised model learns a link between features (the inputs) and a label (the answer). Choosing and preparing good features is often more important than choosing the algorithm, so it pays to understand them well.

Features, labels and the X/y convention

Think of a dataset as a spreadsheet. Each row is one example (also called a sample or observation). Each column you feed into the model is a feature. The column you want to predict is the label, also called the target.

By convention, the feature table is called X (capital, because it's a 2-D matrix) and the labels are called y (lowercase, a 1-D list). Here is a small dataset of rental flats:

import pandas as pd

df = pd.DataFrame({
    "size_sqm": [30, 45, 60, 80],
    "bedrooms": [1, 1, 2, 3],
    "town": ["Nairobi", "Kisumu", "Nairobi", "Nakuru"],
    "rent_kes": [15000, 9000, 25000, 30000],
})

X = df.drop(columns="rent_kes")   # features: everything except the label
y = df["rent_kes"]                # label

print(X.shape)
print(y.tolist())
(4, 3) [15000, 9000, 25000, 30000]

X has 4 rows and 3 feature columns. Since the label is a number, this is a regression problem.

Features (X) Label (y) size_sqm bedrooms town rent_kes 301Nairobi15000 451Kisumu9000 602Nairobi25000 803Nakuru30000 inputs the model learns from what it predicts
Figure 1. Each row is one flat. The first three columns form X; the rent column is y.

Types of features

TypeExamplesHow models use it
NumericalSize in sqm, age, monthly incomeDirectly, often after scaling
Categorical (nominal)Town, mobile network, crop typeMust be encoded as numbers, e.g. one-hot
OrdinalEducation level, T-shirt size (S, M, L)Encoded as ordered numbers (0, 1, 2)
BooleanHas a job, owns a smartphoneAs 0 and 1
Text, images, datesReviews, photos, sign-up dateConverted into numeric features first

Encoding categorical features

Models do maths, so they need numbers. You can't just replace Nairobi with 1, Kisumu with 2 and Nakuru with 3, because that tells the model Nakuru is "three times" Nairobi. Instead use one-hot encoding: one 0/1 column per category.

X_encoded = pd.get_dummies(X, columns=["town"], dtype=int)
print(X_encoded)
size_sqm bedrooms town_Kisumu town_Nairobi town_Nakuru 0 30 1 0 1 0 1 45 1 1 0 0 2 60 2 0 1 0 3 80 3 0 0 1

Each flat now has a 1 in exactly one town column. In a full scikit-learn project you would usually use OneHotEncoder inside a pipeline instead, because it remembers the categories it saw during training and can safely handle a new town at prediction time with handle_unknown="ignore".

Feature engineering

Feature engineering means creating better inputs from the raw data using your knowledge of the problem. Some examples:

  • From a date of birth, compute age.
  • From a transaction timestamp, extract hour of day and day of week (fraud often happens at odd hours).
  • From total rent and size, compute rent per square metre.
  • From a list of M-Pesa transactions, compute average monthly inflow.

A good feature makes the pattern easier to see. Removing useless or duplicate features also helps: they add noise and slow training.

What makes a good feature?

  • Related to the label: size clearly relates to rent; the tenant's favourite colour doesn't.
  • Available at prediction time: you must have the value when you actually need a prediction.
  • Reliable: measured consistently and not mostly missing.
  • Fair and lawful to use: some attributes, or close stand-ins for them, can lead to discrimination (see Bias & Fairness).
Common mistake

Target leakage: including a feature that secretly contains the answer. For example, when predicting whether a loan will default, a column like "number of reminder calls made by the recovery team" is only filled in after a borrower falls behind. The model will look brilliant in testing and fail in real life, because that information doesn't exist when the loan is approved.

Key points

  • Features (X) are the inputs; the label (y) is what you predict.
  • Categorical features need encoding; one-hot encoding avoids fake ordering.
  • Feature engineering turns raw data into more useful inputs.
  • Only use features that will be available, honestly, at prediction time.
Up next · Data PreparationCleaning Data for MLMissing values, outliers and duplicates.
→

Tip: use the ← and → keys to move between pages.