Nginx Reverse Proxy

What you'll explore

1. 01

   Workers, and What "auto" Resolves To

   nginx's workers are processes, forked from a master, each running its own epoll loop over its own connection array — allocated once per worker at ngx_event_process_init() (src/event/ngx_event.c:754) and sized by worker_connections (src/event/ngx_event.c:1084, default 512 at src/event/ngx_event.c:13). Two consequences follow that surprise people. First, worker_processes auto is not container-aware: it resolves to ngx_ncpu (src/core/nginx.c:1567), which is sysconf(_SC_NPROCESSORS_ONLN) (src/os/unix/ngx_posix_init.c:63) — the number of CPUs online on the HOST. A pod with a 0.5-CPU quota on a 64-core node gets 64 workers, each with its own connection array and its own share of memory, competing for half a core. Envoy derives the same number differently: min(hardware threads, sched_getaffinity, cgroup CPU limit) at source/server/options_impl_platform_linux.cc:68, with cgroup detection on by default (:52). Second, the classic "nginx stops accepting when the worker is 7/8 full" never happens in a default build — but the tidy correction, that the counter is simply unread, is wrong too. ngx_accept_disabled (src/event/ngx_event_accept.c:139) has TWO consumers. The famous one — decrement the counter and skip the accept mutex this round — sits inside if (ngx_use_accept_mutex) (src/event/ngx_event.c:219-221) and never runs, because ngx_conf_init_value(ecf->accept_mutex, 0) (:1369) leaves that false. The other one does run: ngx_reorder_accept_events() (src/event/ngx_event_accept.c:450), called after every accept batch (:339), returns early only while c->requests++ % 16 != 0 AND ngx_accept_disabled <= 0 (:476-480), so a worker past the mark re-arms its listening socket immediately (:482-492) instead of waiting for the every-16th rotation — which under EPOLLEXCLUSIVE hands the next wakeup to another worker. That path needs epoll, no reuseport, accept_mutex off and worker_processes > 1 (src/event/ngx_event.c:907-926). So at 7/8 a modern multi-worker nginx does not stop accepting; it starts sharing the listener sooner. Neither consumer CLOSES anything — that is a different watermark in a different file, and it is in the next section.

2. 02

   Two Slots Per Proxied Request

   worker_connections is not a client limit — it is a SLOT limit, and a proxied request holds two of them. The upstream socket is taken from the same per-worker array as the client socket: ngx_event_connect_peer() calls ngx_get_connection() at src/event/ngx_event_connect.c:53, the same allocator the listener uses. So the ceiling a reverse proxy actually has is worker_processes x worker_connections slots, of which each in-flight proxied request consumes two, plus one slot per listening socket and one for the channel (src/event/ngx_event.c:447-460 refuses to start otherwise). When the free list runs low, nginx does not refuse — it REAPS: ngx_drain_connections() (src/core/ngx_connection.c:1405) fires below a 1/16 watermark (:1411) and closes up to max(min(32, reusable/8), 1) idle connections per call (:1427), logging "worker_connections are not enough, reusing connections" (:1421). Only when the free list is empty does ngx_get_connection() return NULL (:1229). Upstream keepalive changes the arithmetic in both directions. Since nginx 1.29.7 it is ON by default for explicit upstream blocks — 32 cached connections per worker, scoped local (src/http/modules/ngx_http_upstream_keepalive_module.c:552-554) — and proxy_http_version now defaults to 1.1 (src/http/modules/ngx_http_proxy_module.c:4032), which means the decade-old recipe of setting proxy_http_version 1.1 and clearing the Connection header is now describing the default. Those cached connections are held per worker, in the config pool (:563) rather than in shared memory, so the fleet-wide pool is keepalive x worker_processes; and when the cache is full, releasing a connection EVICTS the oldest one to make room (:334-341) rather than declining to cache the new one.

3. 03

   keys_zone Is a Census, Not the Cache

   proxy_cache_path names two different stores and only one of them holds responses. The bodies live on disk; keys_zone is a shared-memory index of what is on disk, and it is the half that runs out first, because it is bounded by a number you chose and the disk half is bounded by max_size. Every cached entry takes one fixed-size node: ngx_http_file_cache_exists() allocates sizeof(ngx_http_file_cache_node_t) from the zone's slab (src/http/ngx_http_file_cache.c:930), which is 120 bytes on a 64-bit build and therefore occupies a 128-byte slab class, because ngx_slab_alloc_locked() rounds every small allocation up to a power of two (src/core/ngx_slab.c:210). That is the whole of the "a megabyte holds about eight thousand keys" folklore, derived rather than quoted. The zone is shared across workers, so a hit found by worker 3 is visible to worker 7. Entries are evicted by inactivity, not by TTL: fcn->expire is refreshed to now + cache->inactive on every touch (:977), inactive defaults to 600 seconds (:2407), and a response with a long Cache-Control lifetime that nobody requests for ten minutes is gone regardless. When the zone fills, allocation fails and nginx forces expiry rather than returning an error to the client (:932-941) — the failure mode is a cache that quietly stops caching, which is why the keys_zone gauge deserves a dashboard row of its own.

4. 04

   Terminating TLS Is Per-Connection Work

   TLS termination is the one proxy cost that scales with CONNECTIONS rather than requests, which is why client keepalive and session reuse move the CPU curve more than any cipher choice. nginx stores resumable sessions in a shared-memory zone so that a client landing on a different worker can still resume: ngx_ssl_new_session() (src/event/ngx_event_openssl.c:4378) takes the zone's slab lock and makes TWO allocations per session — the ngx_ssl_sess_id_t index entry (:4447) and a separate copy of the serialized session blob (:4464) — then inserts into an rbtree keyed by session id (:4498) and an expiry queue (:4496), with the lifetime taken from SSL_CTX_get_timeout() (:4494). Set ssl_session_cache to zero and that path is simply never taken: every connection pays a full handshake. Two caveats that keep this honest. The shared cache is a TLS 1.2-era session-ID mechanism; TLS 1.3 resumption normally rides session TICKETS, which are stateless and are not in this zone at all. And the per-session byte cost depends on the serialized blob length (:4444), so the sessions-per-megabyte figure this module shows is a calibrated estimate, not a source constant — it is labelled as such where it appears.

5. 05

   A Leaky Bucket in Shared Memory

   limit_req is a leaky bucket, and the bucket has to live where every worker can see it, because a client's requests are spread across workers by the kernel's accept balancing. That is why limit_req_zone takes a zone name and a size: ngx_http_limit_req_zone() registers it via ngx_shared_memory_add() (src/http/modules/ngx_http_limit_req_module.c:941) and refuses anything under eight pages (:894-897). Per tracked key, nginx allocates offsetof(ngx_rbtree_node_t, color) + offsetof(ngx_http_limit_req_node_t, data) + key length (:488-490) — 32 + 48 + key bytes on a 64-bit build, so an IPv4 $binary_remote_addr state is 84 bytes, rounded by the slab allocator to a 128-byte class (src/core/ngx_slab.c:210). The rate itself is stored in thousandths of a request per second (:939), and the bucket drains by arithmetic rather than by a timer: on each request, excess = previous excess - rate x elapsed_ms / 1000 + 1000 (:454). A per-worker limiter would divide the configured rate by the number of workers in the best case and, in the general case, would let a client through once per worker; nothing about the rate directive would look wrong while the limit silently was not one.