Logs are Your Forensic Evidence: Structured Security Logging

At 3am, when the incident commander asks "who accessed this customer record between 14:00 and 16:00 UTC?", you do not want the answer to depend on grepping through plaintext logs and praying the timestamps are parseable.

Structured logs turn that question into a one-line CloudWatch Insights query.

The minimum useful schema

Every log line your services emit should be JSON, and every line should contain:

{
  "ts": "2025-05-12T14:31:08.221Z",
  "level": "info",
  "service": "billing-api",
  "version": "1.2.3",
  "trace_id": "5f2c…",
  "user_sub": "auth0|abc123",
  "tenant_id": "acme-corp",
  "action": "customer.read",
  "resource_id": "cust_9182",
  "result": "success",
  "ip": "203.0.113.42"
}

ts, service, trace_id, user_sub, action, resource_id, result are the seven fields that turn logs into forensic evidence.

The query that earns its keep

fields @timestamp, user_sub, action, resource_id, result
| filter resource_id = "cust_9182"
| filter @timestamp > 1715520000
| sort @timestamp asc

That's the answer to "who touched this record." It runs in seconds across millions of log lines because the fields are structured, not regexed out of free text.

What good looks like in code

Use a logger that emits JSON by default — structlog in Python, pino in Node, zerolog in Go. Inject trace_id, user_sub, tenant_id once at the request boundary via middleware. Every subsequent log line in that request inherits them automatically.

DO log: every auth decision, every PII read, every privileged action. DO NOT log: passwords, tokens, full prompt bodies (hash them), or full request bodies that may contain PII (log the field names only).

The audit conversation

• "Show me every action user X took in the last 30 days." → one query.
• "Show me every access to PII for tenant Y." → one query, because you tagged action = "pii.*".
• "Prove this log line has not been tampered with." → CloudWatch Logs immutability + log group archive to S3 with Object Lock.

Plaintext logs are a liability. Structured logs are evidence. Treat the schema as a security control, not an afterthought.

Further reading: CloudWatch Logs Insights.

A note on retention

Keep hot logs in CloudWatch for 30 days for fast queries; archive to S3 with Object Lock and a 7-year retention for the compliance regimes that demand it. The S3 copy is your evidentiary record — immutable, cheap, queryable via Athena when needed.