CDN & Edge Caching

What you'll explore

1. 01

   Five Outcomes, Not Two

   A cache lookup does not return hit or miss. HSH_Lookup() (bin/varnishd/cache/cache_hash.c:429) has five exits and the three people never name are where the interesting behaviour lives. HSH_HIT (:618) is the fresh copy. HSH_MISS (:656) is nothing usable, fetch it. But HSH_HITMISS (:612, :634) says an object exists and is marked hit-for-miss — a previous fetch was uncacheable, so the result is remembered as "do not even try to cache this" and each request is passed through separately. HSH_GRACE (:652, :667) returns an EXPIRED copy on purpose. And HSH_BUSY (:691) returns nothing at all yet: another request is already fetching this key, and this one has been parked. Which exit you take decides whether the client waits for your origin, waits for someone else's fetch, or gets served immediately from a copy that is already stale — and only one of the five is the "miss" people budget for. Freshness comes from the response, and when nothing supplies it Varnish uses default_ttl, which is 2m (include/tbl/params.h:472-476) — not infinity, and not zero. Setting ttl to 0 here does not produce a slow cache; it produces a cache that can never take the HSH_HIT exit at all, which is why the hit ratio you asked for stops being reachable.

2. 02

   The Key Is Not The URL

   Varnish's default hash is eight lines of VCL you can read: vcl_builtin_hash (bin/varnishd/builtin.vcl:121-128) calls hash_data(req.url), then hash_data(req.http.host) — falling back to server.ip when there is no Host header. So the default key is URL *plus Host*, which is why one origin serving two hostnames does not share a cache entry between them, and why "the cache key is the URL" quietly breaks the first time someone adds a vanity domain. Vary is the other half. Each distinct combination of the varied header values becomes its own stored object, matched by VRY_Match() (bin/varnishd/cache/cache_vary.c:290) against the vary string built at fetch time by VRY_Create() (:80). This is not a cost you can reason about per-header: the variants MULTIPLY, and every variant needs its own backend fetch to warm. The edge-compute layer has the sharpest statement of the consequence anywhere in these three trees, and it is a comment in Cloudflare's own runtime — ignoreVary is marked jsg::WontImplement (src/workerd/api/cache.h:29) with the reason given at :26-28: "Historically, Cloudflare has not supported the Vary header because it's easy to blow up your cache keys." The same file refuses ignoreSearch (:24) because "Our cache does not support matching without query parameters at match time" (:22-23) — at the edge the query string is part of the key and you cannot ask it not to be.

3. 03

   Purge Is Keyed, Ban Is A List

   Core Varnish has two invalidation mechanisms and their costs are not comparable. VRT_purge() (bin/varnishd/cache/cache_vrt.c:1024) is legal only inside vcl_hit or vcl_miss (:1029-1034) — precisely because it operates on the objhead the request has ALREADY looked up (:1038-1039). It is a hash operation: this key, all of its variants, gone. A ban is the opposite shape. It is an expression appended to a list, and nothing is deleted when you add it; instead every object pays for it later, lazily, in BAN_CheckObject() (bin/varnishd/cache/cache_ban.c:642). The fast path is a pointer compare — if the object's ban is the head of the list there is nothing to test (:664-665) — but otherwise the worker walks every ban newer than that object (:688-694), skipping completed ones (:690-691), and the tests are counted into bans_tested and bans_tests_tested (:698-699). So a ban is cheap to issue and is charged to your hit path, per object, until the ban lurker retires it: the lurker examines ban_lurker_batch objects (1000, include/tbl/params.h:200) then sleeps ban_lurker_sleep (0.010s, :213), which is a ceiling of about 100,000 objects per second regardless of how many you have cached. The folklore that "tag purge scales and URL purge does not" has the core-Varnish mechanics backwards. Tag invalidation scales when a surrogate-key INDEX exists to make it a keyed lookup — and that index is not in this tree: a whole-tree grep finds xkey only in doc/sphinx/reference/vmod.rst:886-898, as an out-of-tree vmod whose own counter is described as "Number of surrogate keys".

4. 04

   What The Runtime Will Not Do For You

   ⚠️ Read the seam first: workerd is the open-source runtime that executes a Worker. It is NOT the Cloudflare production edge, which adds colo routing, tiered cache and the CDN itself — none of which exist in any public tree. Everything below is a RUNTIME fact with a file and a line; anything about how the edge routes or tiers is not cited here at all, because it cannot be. The runtime's Cache API is defined by what it refuses. caches.default is a real handle (JSG_READONLY_INSTANCE_PROPERTY(default, getDefault), src/workerd/api/cache.h:133) with match, put and delete — and then match/has/delete/keys on CacheStorage itself are all jsg::WontImplement (:113, :116, :119, :122) because it "does not support namespace enumeration" (:111). matchAll is refused too, since the cache "does not support one-to-many matching" (:59-60) — which means there is no operation in this API that returns many objects for one criterion, so tag-style invalidation cannot be expressed here either. Non-GET requests return undefined or false unless ignoreMethod is set (:16-20). The other thing a Worker cannot outrun is its own budget: LimitEnforcer (src/workerd/io/limit-enforcer.h:131) gates every outbound call through newSubrequest(bool isInHouse) (:152), which throws once the allowance is spent (doc :146-151, with in-house calls accounted separately from external ones), and exitJs() (:77) reports whether "the isolate has exceeded limits and become condemned" (:76). Moving logic to the edge moves it inside those limits.

5. 05

   The Stampede Is A Default, Not An Accident

   Varnish protects your origin from a stampede without being configured to. And it is a DEFAULT rather than an invariant, which is worth knowing before you rely on it: the scan that finds the busy object checks one flag first — if (oc->flags & OC_F_BUSY) { if (req->hash_ignore_busy) continue; ... } (bin/varnishd/cache/cache_hash.c:525-527, and the branch continues past that test with the busy object's own vary match at :529-533 before setting busy_found at :535). With req.hash_ignore_busy set in VCL a busy object is SKIPPED rather than joined, so busy_found is never set and the request never reaches the waiting list at all — which is why the AZ(req->hash_ignore_busy) assert just after the insert (:673) can hold. Its documented default is false (doc/sphinx/reference/vcl_var.rst:328), and the documented reason to change it is narrower and more interesting than a performance trade: "You only want to do this when you have two server looking up content sideways from each other to avoid deadlocks" (:332-333). One more property worth having, because it is the opposite of what you would guess: the flag is NOT reset when a request restarts. cnt_recv_prep() zeroes it inside if (req->restarts == 0) (bin/varnishd/cache/cache_req_fsm.c:903, :920), so it is initialised on the first pass only and deliberately survives a return(restart). So the honest form of the claim is that Varnish coalesces unless you opt out, and nginx does not unless you opt in. When a request finds a busy object and has no usable stale copy, it is not sent to the backend: VTAILQ_INSERT_TAIL(&oh->waitinglist, req, w_list) (bin/varnishd/cache/cache_hash.c:671, under the comment "There are one or more busy objects, wait for them" at :670) parks it, HSH_BUSY is returned (:691), and one fetch serves all of them. nginx, doing the same job, does the opposite by default: proxy_cache_lock merges to 0 (src/http/modules/ngx_http_proxy_module.c:3892-3893). Off. Every concurrent miss on the same key becomes its own upstream request. Turning it on bounds the herd but not the wait — proxy_cache_lock_timeout and proxy_cache_lock_age both default to 5000ms (:3895-3899), and when the timeout expires the waiting request is sent upstream anyway. The same asymmetry runs through stale-while-revalidate. In Varnish it is grace, and it is CONDITIONAL: the HSH_GRACE exits require an expired object to exist — if (exp_oc != NULL) (:645) and EXP_Ttl_grace(req, exp_oc) >= req->t_req (:648, :660) — with the comment at :661 stating the payoff outright, "we do not wait on the busy object if in grace". default_grace is 10s (include/tbl/params.h:437-441), and default_keep is 0s (:454), so an object is gone at ttl+grace+keep and a request for it starts cold. nginx's equivalents, proxy_cache_use_stale and proxy_cache_background_update, are both off by default (:3859-3862, :3907-3908). So "the CDN absorbs the miss" is true of one of these two out of the box and false of the other, and a shield tier changes the fan-in only where coalescing is already happening.