The Data Science Workflow
A data science project is not a single clever line of code. It is a series of steps, from asking a question to sharing an answer, and knowing those steps keeps your work focused and trustworthy.
The six steps
- Ask a question. Start with a specific, useful question. "Analyse our sales" is vague; "Which three products brought in the most revenue last quarter?" is clear.
- Collect the data. Find data that can answer the question: a shop's sales spreadsheet, a database export, a survey, or public data such as statistics published by the Kenya National Bureau of Statistics.
- Clean the data. Fix missing values, duplicates, typos and wrong data types. Messy data gives wrong answers.
- Explore the data. Look at it: counts, averages, ranges and quick charts. Exploration often reveals problems or surprises.
- Analyse or model. Group, compare and test. Sometimes this step includes a statistical test or a machine learning model.
- Communicate the results. Share a clear answer, the evidence behind it and any limits, usually with a chart or two.
The process is a loop, not a straight line. Exploring data often sends you back to clean it again, and a result frequently leads to a better follow-up question.
A worked example: matatu routes
A SACCO manager asks: "Which of our two routes earns more per trip?" The SACCO has recorded the total fares collected on five trips. One trip's total was never written down.
import pandas as pd
# Step 2: collect the data
trips = pd.DataFrame({
"route": ["CBD-Rongai", "CBD-Thika", "CBD-Rongai", "CBD-Thika", "CBD-Rongai"],
"fare_total": [4200, 5600, None, 6000, 3900], # KES per trip
})
# Step 3: clean - drop the trip with no recorded total
trips = trips.dropna()
# Steps 4 and 5: explore and analyse
averages = trips.groupby("route")["fare_total"].mean()
print(averages)
# Step 6: communicate
best = averages.idxmax()
print(f"{best} earns the most per trip: KES {averages.max():,.0f}")
Notice the averages are shown with .0: because one value was missing, pandas stored the column as decimal numbers (float64). That small detail is typical of real data work.
Five trips is far too few to be confident. A good analyst would say "Thika looks better so far, but we need more trips to be sure." Always mention how much data your answer is based on. The Hypothesis Testing page shows how to check whether a difference could just be chance.
What each step looks like in Python
| Step | Typical tools | Covered on |
|---|---|---|
| Collect | pd.read_csv(), pd.read_excel() | Loading Data |
| Clean | dropna(), fillna(), drop_duplicates() | Cleaning Data |
| Explore | head(), describe(), quick charts | Descriptive Statistics |
| Analyse | groupby(), filtering, SciPy tests | Grouping & Aggregation |
| Communicate | Matplotlib, seaborn, a short written summary | Choosing the Right Chart |
Frameworks you may hear about
Companies often describe this workflow with a named framework. The best known is CRISP-DM (Cross-Industry Standard Process for Data Mining), which uses six phases: business understanding, data understanding, data preparation, modelling, evaluation and deployment. The names differ, but the idea is the same as the six steps above: understand the problem, prepare the data, analyse it, check the results and put them to use.
Habits of good analysts
- Start from the question, not the data. Otherwise you produce charts nobody needs.
- Keep the raw data untouched. Clean a copy, so you can always start again.
- Write code, not manual edits. A script can be rerun next month with new data; hand edits in Excel cannot.
- Sanity-check results. If the average matatu fare comes out as KES 2 million, something went wrong in cleaning.
- Explain limits honestly. Say what the data can't tell you.
Machine learning projects follow a very similar path. Compare this page with The Machine Learning Workflow in the AI/ML series.
Key points
- The workflow is: ask, collect, clean, explore, analyse, communicate.
- It is a loop; you will often go back a step.
- A specific question keeps the whole project focused.
- Always report how much data supports your answer and what its limits are.
Tip: use the ← and → keys to move between pages.
