Optimizing cloud performance requires continuous observability across application health, underlying infrastructure utilization, network responsiveness, and system elasticity.
Tracking these key metrics across five core categories helps identify bottlenecks, optimize user experience, and prevent over-provisioning.
1. The Four “Golden Signals” (Application Level)
Originated by SRE teams at Google, these four primary metrics provide an immediate snapshot of end-user experience and application health.
| Metric | What It Measures | Optimization Signal |
| Latency | Time taken to service a request (e.g., HTTP Response Time, TTFB). | High 95th/99th percentile latency indicates resource contention, unindexed DB queries, or network bottlenecks. |
| Traffic / Throughput | Demand placed on the system (e.g., Requests Per Second, RPS, or HTTP Transactions). | Sudden spikes signal scale-out needs; dropping traffic indicates upstream DNS/load-balancer issues. |
| Errors (Error Rate) | Percentage of failed requests (e.g., HTTP 5xx codes or unhandled exceptions). | Spike in errors indicates infrastructure exhaustion, failing dependencies, or bad deployments. |
| Saturation | How “full” a service is relative to its maximum capacity. | High saturation (>80-90%) precedes sudden latency spikes and dropped requests. |
2. Infrastructure & Compute Utilization Metrics
Monitoring resource consumption helps balance workload performance against cloud infrastructure costs.
-
CPU Utilization: Percentage of total available compute capacity consumed.
-
Target: 60% – 75% average load under peak traffic.
-
Signal: $>85\%$ sustained indicates compute bottlenecks requiring scaling; $<20\%$ sustained signals over-provisioned waste.
-
-
Memory Usage & Swap: Percentage of allocated RAM consumed.
-
Signal: Memory pressure near 100% causes disk swapping or OOM (Out Of Memory) container/VM crashes.
-
-
Network Latency & Bandwidth Throughput: Ingress and egress network data transfer rates (Mbps/Gbps) and packet transfer delay.
-
Signal: High cross-AZ or cross-region network latency points to inefficient inter-service communications.
-
3. Storage & Database Metrics
Database and object storage bottlenecks directly degrade application response times.
+--------------------------------------------------------------------------+
| STORAGE PERFORMANCE METRICS |
+--------------------------------------------------------------------------+
| IOPS --> Read/Write operations completed per second |
| Disk Throughput --> Megabytes per second (MB/s) transferred to/from disk|
| Queue Depth --> Number of I/O requests waiting to be serviced |
+--------------------------------------------------------------------------+
-
IOPS (Input/Output Operations Per Second): Rate of read/write transactions processed by cloud volumes (EBS, Managed Disks). Exceeding IOPS limits results in I/O throttling.
-
Buffer Cache Hit Ratio: Percentage of database queries served from memory rather than disk storage.
-
Target: >95% hit ratio for relational databases (e.g., PostgreSQL, MySQL).
-
-
Database Active Connections & Query Duration: Number of open database handles and execution time for slow queries.
4. Elasticity & Auto-Scaling Metrics
Cloud performance depends on how quickly and efficiently infrastructure adjusts to fluctuating demand.
-
Scale-Out / Provisioning Time: Time elapsed between an auto-scaling trigger and the new instance/pod becoming fully healthy to accept live traffic.
-
Scaling Flapping / Thrashing: Rapid, repetitive scaling up and scaling down within short timeframes, which leads to unstable performance and unnecessary cloud costs.
-
Cold Start Latency: Delay experienced when invoking serverless functions (e.g., AWS Lambda, Azure Functions) from an inactive state.
5. Unit Economics & FinOps Efficiency Metrics
Performance optimization is incomplete without correlating system responsiveness with cloud expenditure.
Unit Cost Principle: True performance optimization makes applications faster and more cost-efficient per transaction processed.
-
Cost Per Transaction / Request: Total infrastructure cost divided by the number of successful user requests handled.
-
Idle Resource Ratio: Percentage of provisioned CPU cores and RAM allocated to running workloads that sit unutilized during operational hours.
Metric Monitoring Threshold Matrix
| Category | Primary Metric | Warning Threshold | Critical Threshold |
| Compute | CPU / RAM Utilization | > 75% sustained | > 90% sustained |
| User Experience | Latency (p99) | > 500 ms | > 2,000 ms |
| Reliability | HTTP 5xx Error Rate | > 1% of traffic | > 5% of traffic |
| Database | IOPS Utilization | > 80% volume limit | 100% (Throttled) |
| Storage | Disk Space Free | < 20% available | < 10% available |