Track resource usage

Start tracking CPU, memory, disk I/O, and network per container on prod-app namespace. Deploy the monitoring agent to all three app clusters, ensure metrics are retained for 30 days, set alerts for CPU ≥ 85% sustained 5 minutes and disk I/O latency > 20ms, and verify dashboards show historical spikes for host db-02 and pod api-7.

Before I present resource metrics to ops, make them easy to act on: surface the top three hosts by sustained CPU, highlight pods with memory growth over 24 hours, show which alerts are firing most often, and call out any missing instrumentation. Flag anything that would make a reviewer hesitate to approve scaling.

At 03:12 alerts for high CPU and OOMs hit on prod and I couldn’t tell if a deploy or a traffic spike caused it. I have timelines, but not which metric tripped first. Recommend the fastest way to determine causation from the current data: which query or chart to run, whether to blame autoscaler or a leak, and the safe immediate action to stop customer impact.

Every few weeks an alert fires, we scale or restart a service, and later the same issue returns. I spend cycles firefighting instead of eradicating the cause. What three habits should my team adopt — in instrumentation, runbooks, and postmortems — to turn these recurring spikes into permanent fixes?