Skip to content
Kuza Ndoto Academy

Monitoring with CloudWatch

Beginner → IntermediateChapter 7 · Page 24 of 304 min readUpdated 15 Nov 2026

Amazon CloudWatch is AWS's monitoring service. It collects metrics (numbers over time, such as CPU use), stores logs from your apps, raises alarms when something crosses a threshold and shows it all on dashboards. Without monitoring, you only find out about problems when users complain.

Metrics

A metric is a time-ordered series of data points. Each metric is identified by:

  • a namespace, such as AWS/EC2, AWS/Lambda or your own KuzaShop;
  • a metric name, such as CPUUtilization, Invocations or Errors;
  • dimensions, name-value pairs that say which resource, such as InstanceId=i-0abc1234def567890.

You view metrics as statistics (Average, Sum, Minimum, Maximum, percentiles like p99) over a period (for example 60 or 300 seconds). Many AWS services publish metrics automatically. EC2 sends basic metrics every five minutes, or every minute with detailed monitoring (an extra charge). Note that EC2 cannot see inside your instance, so memory and disk-space usage need the CloudWatch agent installed.

Your app can publish custom metrics too, such as the number of payments received:

import boto3

cloudwatch = boto3.client("cloudwatch")
cloudwatch.put_metric_data(
    Namespace="KuzaShop",
    MetricData=[
        {
            "MetricName": "PaymentsReceived",
            "Dimensions": [{"Name": "Channel", "Value": "mobile-money"}],
            "Value": 1,
            "Unit": "Count",
        }
    ],
)

Logs

CloudWatch Logs stores log lines in log groups (usually one per app or function, such as /aws/lambda/payments-notify), each containing log streams (one per instance or container). Lambda sends everything you print there automatically; EC2 needs the CloudWatch agent; ECS uses the awslogs driver.

Watch a function's logs live from the CLI (example output):

aws logs tail /aws/lambda/payments-notify --follow --since 10m
2026-11-14T08:01:12.345000+00:00 2026/11/14/[$LATEST]5f2c9a START RequestId: 8f1d2c3e-1111-4a5b-9c7d-0e1f2a3b4c5d 2026-11-14T08:01:12.349000+00:00 2026/11/14/[$LATEST]5f2c9a Payment of KES 1500 received from 0000 2026-11-14T08:01:12.352000+00:00 2026/11/14/[$LATEST]5f2c9a END RequestId: 8f1d2c3e-1111-4a5b-9c7d-0e1f2a3b4c5d

CloudWatch Logs Insights lets you search and analyse logs with a query language. This finds the 20 most recent errors:

fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc
| limit 20
Warning

By default, log groups keep logs forever, and you pay for log storage every month as well as for ingesting logs. Set a retention period on every log group, for example 14 or 30 days for practice projects:

aws logs put-retention-policy --log-group-name /aws/lambda/payments-notify \
    --retention-in-days 14

Alarms

An alarm watches one metric (or a calculation on several) and changes state when a condition holds for a number of periods. Its three states are OK, ALARM and INSUFFICIENT_DATA. When the state changes, the alarm can take actions: notify an Amazon SNS topic (which sends email, SMS or calls a webhook), trigger an Auto Scaling policy, or stop, reboot or recover an EC2 instance.

EC2, RDS,Lambda MetricCPUUtilization AlarmCPU > 80%for 2 x 5 min ALARM SNS topicops-alerts Email or SMS to on-call Auto Scaling action
Figure 1. Metrics flow into an alarm. When CPU stays above 80% for two consecutive five-minute periods, the alarm enters ALARM and can notify people through SNS and trigger automatic actions.

Set up the alarm in Figure 1 for the results portal's Auto Scaling group. First create an SNS topic and subscribe your email (you must click the confirmation link AWS emails you):

aws sns create-topic --name ops-alerts
{ "TopicArn": "arn:aws:sns:eu-west-1:123456789012:ops-alerts" }
aws sns subscribe --topic-arn arn:aws:sns:eu-west-1:123456789012:ops-alerts \
    --protocol email --notification-endpoint you@example.com

aws cloudwatch put-metric-alarm --alarm-name results-web-high-cpu \
    --namespace AWS/EC2 --metric-name CPUUtilization \
    --dimensions Name=AutoScalingGroupName,Value=results-asg \
    --statistic Average --period 300 --evaluation-periods 2 \
    --threshold 80 --comparison-operator GreaterThanThreshold \
    --treat-missing-data notBreaching \
    --alarm-actions arn:aws:sns:eu-west-1:123456789012:ops-alerts

Good alarms watch things users feel: error rates (Lambda Errors, ALB HTTPCode_Target_5XX_Count), latency (TargetResponseTime p99) and queues backing up, not just CPU.

Tip

For billing, prefer AWS Budgets (see Cost Management). CloudWatch also has an EstimatedCharges billing metric, but it is only in us-east-1 and must be enabled in Billing preferences first.

Dashboards, and how CloudWatch differs from CloudTrail

Dashboards put graphs and alarm states for many resources on one screen, useful on results day or during a sale. Do not confuse CloudWatch with CloudTrail:

CloudWatchCloudTrail
How are my resources and apps performing?Who did what in my account, and when?
Metrics, logs, alarms, dashboardsA record of API calls (for example "amina terminated i-0abc... at 14:02")

Costs

Many basic metrics are included, but custom metrics, detailed monitoring, alarms, dashboards, log ingestion, log storage and Logs Insights queries are all charged. Check the CloudWatch pricing page, set log retention and delete alarms and dashboards from finished experiments.

Key points

  • Metrics are identified by namespace, name and dimensions, and viewed as statistics over a period.
  • CloudWatch Logs organises logs into groups and streams; set a retention period on every group.
  • Alarms move between OK, ALARM and INSUFFICIENT_DATA and can notify SNS or trigger scaling.
  • Memory and disk metrics for EC2 need the CloudWatch agent.
  • CloudWatch monitors performance; CloudTrail records who did what.
Up next · Operations & CostInfrastructure as Code: CloudFormation & CDKDefine your cloud resources in code.
→

Tip: use the ← and → keys to move between pages.