How the nginx load balancer routes traffic between app containers

Understand the Nginx upstream configuration, round-robin routing, health checks, and how Nginx handles a failed app container.

The upstream block

In the Nginx config, an upstream block defines the pool of app containers that can receive traffic. Each container's internal address (e.g. app-ha-1:3000, app-ha-2:3000) is listed as a server directive. Nginx automatically adds or removes servers from this list when the autoscaler updates the config and sends a SIGHUP reload signal.

Round-robin load balancing

By default, Nginx distributes incoming requests across app containers in round-robin order: request 1 goes to app-ha-1, request 2 to app-ha-2, request 3 back to app-ha-1, and so on. Because the app containers are stateless, any container can handle any request without needing to share session data.

Active health checks

Nginx runs active health checks against each upstream server by periodically sending a GET request to /api/health on each app container. A 200 response means the container is healthy. Two consecutive failures (configurable) cause Nginx to mark the container as unavailable and stop routing traffic to it.

What happens when an app container goes down

If an app container crashes or stops responding: 1. Nginx's health check fails for that container 2. After the failure threshold is reached, Nginx removes the container from the routing pool 3. All new requests are sent only to the remaining healthy containers 4. The failed container is still listed in the config but receives no traffic 5. When the container recovers and passes health checks, Nginx automatically adds it back to the pool

Zero-downtime config reloads

When the autoscaler adds or removes containers, it updates the Nginx config file and sends SIGHUP to the Nginx master process. Nginx reloads the config gracefully — in-flight requests on the old worker processes complete normally before those workers exit. New requests are handled by new workers with the updated config. Users experience no interruption.