Cache Stampede Protection With TaskFused, One Rebuild When a Busy Page Expires

A news site’s front page is built from a dozen slow queries, so it’s cached for five minutes. At the moment the cache entry expires, three hundred readers are loading the page. Every one of them misses the cache, and every one starts the same dozen queries. The database, which was fine a second ago, is suddenly doing three hundred times the work, pages time out, and the timeouts make readers reload. That’s a cache stampede, sometimes called the thundering herd.

The usual cure is to let one request rebuild the page while the rest wait for it. Web caches such as nginx do this with a lock on the cache entry, and in Go, singleflight does it inside one process. TaskFused does the same thing across every process on the machine, and it keeps going if the process doing the rebuild dies.

Put the Window in the Name

TaskFused keeps a finished result until you prune it. It has no expiry time of its own. So the freshness rule goes into the key: round the clock down to the cache window and make that part of the name.

window := time.Now().UTC().Truncate(5 * time.Minute).Format("2006-01-02T15:04")
key := "front-page/" + window // front-page/2026-10-18T09:05

sub, err := store.Submit(ctx, taskfused.Request{
    Namespace: "pages", Handler: "front-page", Version: "v7",
    Key: key, JoinExisting: true,
})
if err != nil {
    return err
}
op, err := store.Wait(ctx, "pages", sub.Operation.ID)
if err != nil || op.State != taskfused.StateSucceeded {
    return previousWindowPage(ctx) // a failed or canceled rebuild is a normal answer, not an error
}

Everyone inside the same five minutes builds the same name. The first request creates the operation, the other 299 join it, and one worker runs the queries. Once it’s done, the rest of the window’s readers get the stored page without waiting. At 09:10 the name changes, and the first reader after that starts the next build.

A normal in-memory cache still sits in front of this. TaskFused takes care of the misses, and keeps them to one build per window however many processes miss at the same moment.

When the Rebuild Crashes

With an in-process lock, a crash during the rebuild is bad timing: the lock goes with the process, and every waiter on that process gets an error. Other processes never knew the rebuild was happening and start their own.

Here the rebuild is an operation in a SQLite file. The worker building the page holds a lease and renews it while it works. If it dies, the lease runs out, another worker picks up the same operation and builds the page, and everyone who was waiting gets that result. Pick a lease a little longer than a slow rebuild usually takes, so a healthy worker is never mistaken for a dead one.

Not Waiting Too Long

A reader shouldn’t wait twenty seconds for a front page. Give the wait a short deadline, and if it runs out, serve the previous window’s page, which is still stored under its old name:

waitCtx, cancel := context.WithTimeout(ctx, 2*time.Second)
defer cancel()
op, err := store.Wait(waitCtx, "pages", sub.Operation.ID)
if err != nil || op.State != taskfused.StateSucceeded {
    return previousWindowPage(ctx) // the 09:00 page, slightly stale but instant
}

Abandoning the wait never cancels the rebuild. It finishes in the background, and the next reader gets the new page.

Tidying Up

Every window leaves a stored page behind, 288 a day for a five-minute window. Run taskfused prune-results on a schedule to remove results older than an hour or a day. Pruning keeps the record of each window, which is small, so set max-operations with that in mind, or start a fresh namespace each month.

The explainer on request coalescing has the background on why this works.