Search
4 results for “observability”
Search results
Observability Fundamentals
How logs, metrics, and traces answer different questions, why monitoring is not debuggability, and the cardinality trap that breaks metrics systems.
What is the difference between monitoring and observability?
Monitoring means watching a predefined set of signals for known failure modes, dashboards and alerts built around questions you already knew to ask ("is CPU above 80%?"). Observability is a property of a system: how well you can answer new, previously-unasked questions about its internal state using only its external outputs (logs, metrics, traces), without shipping new code. Monitoring tells you something is wrong; observability is what lets you figure out why, including for failure modes nobody anticipated when the dashboards were built.
What distinct question does each of logs, metrics, and traces answer?
Metrics answer "what is happening, in aggregate, over time", cheap to store, good for dashboards and alerting thresholds, but they lose individual event detail. Logs answer "what exactly happened in this specific event", full detail but expensive to store and search at scale. Traces answer "where did time go across this one request as it moved through multiple services"; they reconstruct causality and latency across service boundaries that neither logs nor metrics show on their own. A mature observability setup uses all three together, correlated by shared identifiers like a request or trace ID.
What is metric cardinality, and why can it break a monitoring system?
Cardinality is the number of unique label/tag combinations a metric can have. A metric like `http_requests_total{user_id=...}` has cardinality equal to the number of distinct users, potentially millions, because most metrics backends store a separate time series per unique label combination. High-cardinality labels cause a combinatorial explosion in stored time series, which can degrade or crash a metrics backend entirely. The fix is keeping metric labels low-cardinality (route, status code, method) and pushing genuinely high-cardinality data (user IDs, request IDs) into logs or traces instead, where it belongs.