A JWKS cache keyed on the token, not on a timer

A key set cached for fifteen minutes stays correct right up to the moment the issuer rotates. An issuer signs with a new key as soon as it publishes it, so a resource server waiting out its TTL turns away every token minted in the gap: good signature, trusted issuer, correct audience, 401. What the cache needs is a second trigger with nothing to do with age, a token naming a key id the cache does not hold, and once you add that trigger the hard decisions move somewhere new.

I have been verifying bearer tokens since 2018, in production systems where money moves, and this is the failure that never turns up in staging. Rotations are rare, on the issuer's side they are instant, and the symptom is a few minutes of 401s that stop by themselves before anyone has finished reading the logs. The code below comes from mcpaudience, a small Go package that does the resource-server half of MCP authorization, and the key set lives in jwks.go.

The timer answers a question nobody asked

A TTL answers one question: how old is my copy of the key set. The question a verifier actually has is whether that copy contains the key this token names. Those two questions agree nearly all of the time and come apart during exactly one event, a rotation. The issuer publishes rsa-2 and starts signing with it right away, and my cached copy, eight minutes old and perfectly fresh by its own accounting, holds only rsa-1.

So this is not a TTL cache. The lookup takes the kid first and the clock second:

func (j *JWKS) Key(ctx context.Context, kid string) (crypto.PublicKey, error) {
    if err := j.checkURL(); err != nil {
        return nil, err
    }

    // The lock is held across the fetch, so a burst of requests arriving on a
    // rotation makes one request at the issuer rather than one each.
    j.mu.Lock()
    defer j.mu.Unlock()

    cached, known := j.keys[kid]
    if known && !j.stale() {
        return cached, nil
    }
    if err := j.refresh(ctx); err != nil {
        if known {
            // An issuer being unreachable is not evidence that the key it
            // published stopped being its key. Tokens still expire on their own.
            return cached, nil
        }
        return nil, fmt.Errorf("no key for kid %q: %v", kid, err)
    }
    if key, ok := j.keys[kid]; ok {
        return key, nil
    }
    if reason, ok := j.skipped[kid]; ok {
        return nil, fmt.Errorf("the key set lists kid %q but it cannot be used: %s", kid, reason)
    }
    return nil, fmt.Errorf("no key for kid %q in the key set at %s", kid, j.URL)
}

The fast path is those first four lines: a kid I hold, in a set that is not stale, answered out of memory with no network at all. Everything after that is the rotation path. CacheFor is still there and still forces a refetch at fifteen minutes by default, but its job has shrunk to housekeeping, eventually noticing keys the issuer took away. It is not what makes rotation work any more.

About the mutex: it is held across an HTTP request, not only around the map. That serializes verification for the length of one fetch, which sounds expensive until you price the alternative: a rotation throws off a burst of unknown-kid lookups that all arrive together, and with no lock each one of them opens its own connection to the issuer. One bounded wait is cheaper than a herd I caused myself. Bounded is doing real work in that sentence, because the default client carries a ten second timeout and the response body is read through io.LimitReader up to maxKeySetBytes, so a hanging or endless issuer cannot sit on the lock forever.

A fetch the caller can trigger needs a floor

This is the part that makes the unknown-kid trigger more than a one-line change. The kid comes out of the token header, and the token header is attacker-controlled. If an unknown kid means a fetch, then a stream of tokens carrying invented kids means a fetch each, and my resource server turns into a small load generator aimed at my own authorization server. That server is shared infrastructure: the amplification takes out the login flow, not just my service.

So refresh refuses to run twice inside MinRefresh, thirty seconds by default, and it records the attempt before it tries:

func (j *JWKS) refresh(ctx context.Context) error {
    now := j.now()
    if since := now.Sub(j.attempted); !j.attempted.IsZero() && since < j.minRefresh() {
        return fmt.Errorf("the key set was fetched %s ago and is not fetched again more than once every %s",
            since.Round(time.Second), j.minRefresh())
    }
    j.attempted = now

    body, err := j.get(ctx)
    if err != nil {
        return err
    }
    keys, skipped, err := parseKeySet(body)
    if err != nil {
        return fmt.Errorf("%s: %v", j.URL, err)
    }
    j.keys, j.skipped, j.fetched = keys, skipped, now
    return nil
}

func (j *JWKS) minRefresh() time.Duration {
    if j.MinRefresh > 0 {
        return j.MinRefresh
    }
    return 30 * time.Second
}

j.attempted = now sits above j.get, and that is the line I would get wrong if I were writing this in a hurry. Record the attempt only on success and a down issuer means the floor never engages, so every single request retries: the floor would hold exactly when nothing needed holding, then disappear during the outage, when hammering a sick issuer is at its worst. Note too that j.fetched moves forward only on success, so a failed attempt limits retries without making a stale set look fresh. Two timestamps doing two different jobs, and collapsing them into one is tempting and wrong.

The behaviour is cheap to pin down in a test, and this is the one I would write first:

// Refetching on an unknown kid is free traffic at the issuer unless it is rate
// limited: invented key ids must not become one request each.
func TestAnUnknownKeyIDIsNotAFetchPerToken(t *testing.T) {
    s := newSigner(t)
    issuer := newKeyServer(t, rsaJWK("rsa-1", &s.rsaKey.PublicKey))
    v := keySetVerifier(&JWKS{URL: issuer.url, MinRefresh: time.Minute, Now: func() time.Time { return now }})

    for i := 0; i < 5; i++ {
        _, err := v.Verify(context.Background(), s.sign(t, "RS256", "invented", goodClaims()))
        if !errors.Is(err, ErrBadSignature) {
            t.Fatalf("err = %v, want ErrBadSignature", err)
        }
    }
    if got := issuer.count(); got != 1 {
        t.Errorf("the issuer was asked %d times for a key id it never published, want once", got)
    }
}

The injected Now is what makes this testable without sleeping. A key set cache is almost all clock-dependent logic, and a clock you cannot set is a cache you cannot test.

Unknown is three answers, not one

After a successful refresh there are three distinct reasons the kid is still not usable, and the tail of Key keeps them apart. The key is present: return it. The kid is in skipped, meaning the issuer did publish it but parseKeySet dropped it and remembered why: an HMAC or none alg, an RSA modulus under 2048 bits, a curve that is not P-256, a kty of oct. That refusal says so, with the reason attached. Only a kid the issuer has genuinely never published comes back as "unknown".

Two lines of distinction there save an hour of somebody's day. "Unknown kid" sends an operator off to the issuer to check whether the key was published, and the issuer answers yes, it is right there in the document. "The key set lists kid rsa-3 but it cannot be used: RSA modulus is 1024 bits" sends them straight to the right place. A set with no usable key in it is an error rather than an empty cache, and the previous keys stay put, since a malformed publish should not take verification down.

The fetch is a trust boundary too, and the cheapest guard sits at configuration time:

func (j *JWKS) checkURL() error {
    raw := strings.TrimSpace(j.URL)
    if raw == "" {
        return fmt.Errorf("no key set URL configured")
    }
    parsed, err := url.Parse(raw)
    if err != nil || !parsed.IsAbs() || parsed.Host == "" {
        return fmt.Errorf("key set URL %q is not an absolute URL", j.URL)
    }
    if parsed.Scheme != "https" && parsed.Hostname() != "localhost" && parsed.Hostname() != "127.0.0.1" {
        return fmt.Errorf("key set URL %q must use https: a key set an attacker on the path can replace verifies the attacker's tokens", j.URL)
    }
    return nil
}

The error message is the argument. Nothing else in a verifier counts for much if the document that supplies the public keys can be swapped in transit.

Availability over freshness, stated out loud

The last branch is the one worth arguing about. When the refresh fails and the kid was already held, the cached key verifies the token anyway. My reasoning: an issuer being unreachable is not evidence that the key it published stopped being its key, and the tokens it signed expire on their own. Refusing every request while the issuer is down converts a brief issuer blip into a total outage of my service, and it buys no security, since nothing about an HTTP 503 says anything about a public key.

The cost is real and I will name it. A key the issuer deliberately removed keeps verifying tokens until the next successful fetch. If a signing key leaks and the issuer pulls it from the document, my server honours it for as long as the issuer stays unreachable. That is the trade, and I think it is only acceptable because a key set was never a revocation mechanism: pulling a key does not invalidate the already-issued tokens signed with it either. If you need revocation that takes effect now, you need token introspection, and then you are making a request per token with its own failure policy instead of pretending a cached document can do that job.

Note that the fallback applies only to a kid already held. An unknown kid during an outage gets the error, because there is nothing to fall back to and guessing would mean picking some other key out of the set.

One more detail decides what the client sees. The verifier resolves a kid against its configured Keys first and the fetched KeySet second, so a fetched key never overrides a pinned one, and every key-set failure comes back wrapped in ErrBadSignature:

func (v *JWTVerifier) key(ctx context.Context, kid string) (crypto.PublicKey, error) {
    if key, ok := v.Keys[kid]; ok {
        return key, nil
    }
    if v.KeySet == nil {
        return nil, fmt.Errorf("%w: no key for kid %q", ErrBadSignature, kid)
    }
    if strings.TrimSpace(kid) == "" {
        // Without a kid there is nothing to look up and nothing to rotate; a
        // verifier that guesses picks the wrong key the first time one is added.
        return nil, fmt.Errorf("%w: the token names no kid, so no key can be chosen", ErrBadSignature)
    }
    key, err := v.KeySet.Key(ctx, kid)
    if err != nil {
        return nil, fmt.Errorf("%w: %v", ErrBadSignature, err)
    }
    return key, nil
}

The wrapping is a deliberate answer to a question that comes up in review: should "I could not reach the issuer" be a 500? It should not. The client's situation is identical either way, the token it presented did not verify and its move is to go get a new one, and a 500 invites it to retry that same token against a server it has just been told is broken. A token with no kid at all is refused for the same reason it cannot be rotated: there is nothing to look up, and a verifier that guesses picks wrong the first time the issuer adds a second key.

What I would do differently

The rate limit is global, not per kid. An attacker spraying invented kids can burn the one allowed attempt in an interval and delay the refetch that a genuine rotation needs by up to MinRefresh. With a thirty second default that window is small and the failure heals itself, so it is a defensible shape for a small package. In a system where I controlled both ends, I would keep a per-kid negative cache next to the floor, so a kid that was already refused and is already absent from a freshly fetched set does not spend the shared attempt at all.

I would also not leave the outage recovery passive. As it stands, the last good set returns to normal only when some request happens to arrive with a kid that is unknown or a set that has gone stale. A background refresh attempt during a known-failed state would shorten the window in which a removed key stays live, which is the one cost of the availability trade.

And I would instrument skipped and attempted. Both are facts about the issuer that nothing else in the system knows: a key published but unusable, and a refetch declined by the floor. They stay invisible until someone reads a 401 and starts guessing...

The shape that matters generalizes past JWKS. When a cache's correctness depends on an external event rather than on elapsed time, the trigger belongs with the event and not with the clock, and the moment that trigger becomes reachable by an untrusted input, it needs a floor that counts failures.

Originally published on polycratia.com — where I write about payment systems, crypto rails and marketplace backends.