Write Structured Logs You Can Search during an Incident

A practical approach

Step 1

Use a logging library that outputs structured records, and write logs to standard output in containerised environments.

Step 2

Define a small set of standard fields, such as timestamp, level, service, request_id, route and duration_ms, and use them everywhere.

Step 3

Attach the request ID to every log line within a request automatically, using middleware or context propagation.

Step 4

Log events at meaningful points: request completion with status and duration, external calls with outcome and errors with type and context.

Step 5

Exclude secrets and unnecessary personal data, and set log levels so normal operation is not drowned in debug output.

Worked example

Learners report failed payments, but the logs contain lines such as 'payment failed!!' with no identifiers. After switching to structured logs with request_id, route, provider_status and duration_ms, the team filters for route=/checkout and level=error, sees that all failures share one provider status after a configuration change and fixes it within minutes.

A mistake to avoid

Do not log entire request and response bodies by default. They often contain passwords, tokens or personal data, and they make logs expensive to store and slow to search.

Try it yourself

Convert the logging in one service to structured output with five standard fields. Simulate an error and practise finding all log lines for that request by its ID.

Your implementation checklist

• Logs are structured with named fields
• Field names are standardised across services
• Every line in a request carries its request ID
• Key events and errors include useful context
• Secrets and unnecessary personal data are excluded