How to Read a Server Health Check

A green health endpoint answers only the question it was designed to ask. It may prove that a process responds to HTTP while users still cannot log in, save data, or receive a page. Before treating a server health check as evidence of service health, read its contract: what it tests, where the request came from, and which component consumes the result.

Identify the kind of check

Distinguish a process-alive check from a traffic-readiness check. Kubernetes documents liveness, readiness, and startup probes: liveness can trigger a restart, while readiness controls whether a pod receives service traffic. Startup probes can protect slow-starting containers from premature liveness failure. Those behaviors are specific to a configured Kubernetes environment; an arbitrary /health URL does not acquire them automatically.

Ask whether the check tests a dependency. A database connection may be essential for writes but irrelevant to a static page. Including every dependency in a liveness probe can cause restarts during a downstream outage and make recovery harder. Omitting a critical dependency from readiness can send traffic to an instance that cannot serve its core function. Decide by the component's actual role and the consequence of a failed check.

Read the endpoint's status code, body, response time, and timestamp if present. A 200 response with a JSON field saying degraded is not the same as an explicit failure status to a load balancer. Conversely, a temporary non-200 response might indicate a deliberate drain during deployment. Find the consumer's rules rather than inferring behavior from the endpoint name.

Follow the health check's vantage point

Where does the request originate? A local curl to localhost tests a different network path than an external monitor or load balancer. It may bypass TLS termination, DNS, firewall rules, authentication middleware, and the public route. Run a second check from the vantage point relevant to the user report. Keep the local check as evidence about the process, not as proof that the public site works.

AWS documents Application Load Balancer target health checks, including configured protocol, port, path, intervals, and thresholds. It also documents a surprising fail-open behavior when all registered targets in a target group are unhealthy. A status dashboard can therefore show failing target checks while traffic still reaches those targets. Read the particular load balancer's behavior before equating target health with routing.

Inspect the instance identity and deployment revision alongside the result. One healthy target does not prove every replica is healthy. A rolling deployment may have a mixture of versions; a single check can sample only one. If the check is served by a proxy, confirm which upstream answered it. Distinguish a cached response from one generated by the application when caching is possible.

1. Identify the endpoint, caller, target instance, and configured interval.
2. Read status, body, timing, and the exact contract being tested.
3. Check how the orchestrator or load balancer acts on that result.
4. Compare dependency signals and logs for the same time window.
5. Verify a representative user path from the appropriate vantage point.

Interpret changes over time

One failure is not the same as a sustained unhealthy state. Health systems often use consecutive success and failure thresholds to avoid rapid toggling. Match the alert timestamp to the check interval and threshold. A probe that fails once during startup might never remove an instance from service; repeated failures may. Look at a time series and identify whether failures cluster on a single instance, route, or deployment version.

Pair the check with other signals. Logs can show the error that preceded a failure; request metrics can reveal whether users experienced errors or latency; traces can locate a slow dependency. A successful probe with elevated user errors may mean the probe's path is too shallow. A failed deep check while user requests succeed may reflect a noncritical dependency included in the probe. Neither observation alone proves the correct configuration.

When changing a probe, test its failure semantics. What happens if the database is briefly unavailable? Does the instance restart, stop receiving traffic, or merely report degraded status? What happens during planned maintenance and startup? A probe should support recovery and safe routing. It should not create a restart loop because an external service had a short outage.

Hypothetical example: green local check, failed sign-in

Imagine a hypothetical web app with two replicas. An operator runs curl against /health on one container and gets 200 in 12 milliseconds. Users still report failed sign-ins. The endpoint only returns a fixed body after the process starts; it does not test the session store. Logs on the second replica show session-store timeouts, and the public load balancer continues sending it requests because its shallow target check passes.

The team first verifies a safe sign-in test path and checks both replicas. They add a separate readiness signal that reflects the session-store dependency, retain a lightweight liveness check, and test how traffic behaves when the store fails. This hypothetical example does not prescribe that every app make session storage part of readiness. It shows why the meaning and consumer of a check matter more than a green color.

Checklist for an incident note

• Record the exact endpoint, caller, instance, revision, and time.
• State whether it is liveness, readiness, startup, target, or external monitoring.
• Read configured path, status matcher, interval, and thresholds.
• Check the health body and response latency, not just its color.
• Compare all relevant replicas and dependencies.
• Verify a representative user operation through the public path.
• Explain what the check established and what remained untested.

A health check is a deliberately narrow instrument. Read its scope precisely, then combine it with the traffic path and user operation that matters. That yields an incident statement a teammate can act on, rather than a reassuring but incomplete screenshot of a green badge.