Collapse Concurrent Identical Requests in Go with singleflight
package pricecache
import (
"context"
"sync"
"golang.org/x/sync/singleflight"
)
type Loader func(ctx context.Context, symbol string) (float64, error)
type PriceCache struct {
group singleflight.Group
cache sync.Map // symbol -> float64
loader Loader
}
func New(loader Loader) *PriceCache {
return &PriceCache{loader: loader}
}
func (c *PriceCache) Get(ctx context.Context, symbol string) (float64, error) {
if v, ok := c.cache.Load(symbol); ok {
return v.(float64), nil
}
v, err, _ := c.group.Do(symbol, func() (interface{}, error) {
price, err := c.loader(ctx, symbol)
if err != nil {
// Don't let one failure be reused by later requests.
c.group.Forget(symbol)
return nil, err
}
c.cache.Store(symbol, price)
return price, nil
})
if err != nil {
return 0, err
}
return v.(float64), nil
}
func (c *PriceCache) Invalidate(symbol string) {
c.cache.Delete(symbol)
c.group.Forget(symbol)
}
package pricecache
import (
"encoding/json"
"net/http"
)
type PriceHandler struct {
Cache *PriceCache
}
func (h *PriceHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
symbol := r.URL.Query().Get("symbol")
if symbol == "" {
http.Error(w, "missing symbol", http.StatusBadRequest)
return
}
price, err := h.Cache.Get(r.Context(), symbol)
if err != nil {
http.Error(w, "upstream unavailable", http.StatusBadGateway)
return
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"symbol": symbol,
"price": price,
})
}
package pricecache
import (
"context"
"sync"
"sync/atomic"
"testing"
"time"
)
func TestGetCollapsesConcurrentCalls(t *testing.T) {
var calls int64
loader := func(ctx context.Context, symbol string) (float64, error) {
atomic.AddInt64(&calls, 1)
time.Sleep(20 * time.Millisecond) // simulate slow upstream
return 42.0, nil
}
c := New(loader)
const callers = 100
var wg sync.WaitGroup
wg.Add(callers)
for i := 0; i < callers; i++ {
go func() {
defer wg.Done()
price, err := c.Get(context.Background(), "AAPL")
if err != nil || price != 42.0 {
t.Errorf("unexpected result: %v %v", price, err)
}
}()
}
wg.Wait()
if got := atomic.LoadInt64(&calls); got != 1 {
t.Fatalf("expected loader to run once, ran %d times", got)
}
}
This snippet demonstrates the cache-stampede problem and how golang.org/x/sync/singleflight solves it. When many goroutines request the same expensive value at the same moment — a slow database read, an upstream API call, a cache miss under load — the naive approach fires one call per request. Under a traffic spike this can hammer the backing store dozens or hundreds of times for a value that only needed to be computed once. singleflight collapses those duplicate in-flight calls into a single execution and shares the one result (and error) with every caller that was waiting.
In pricecache.go, the PriceCache wraps a singleflight.Group alongside a fast in-memory sync.Map cache and a loader function that performs the real work. Get first checks the local cache; on a hit it returns immediately. On a miss it calls group.Do(key, fn), which guarantees that for a given key only one fn runs at a time while all other concurrent callers block and then receive the same (value, err). The shared boolean returned by Do reports whether the result was delivered to more than one caller, which is useful for metrics. Crucially, once the loader succeeds the value is written to the cache so subsequent calls skip singleflight entirely — singleflight deduplicates the stampede, the cache handles steady-state reads.
The implementation also shows an important pitfall: error caching. Because Do shares the error with every waiter, a single transient failure is amplified across all collapsed callers. The code avoids poisoning the local cache on error and calls group.Forget(key) so the next request retries fresh rather than reusing a possibly-stale in-flight slot. Using DoChan instead of Do would allow a context timeout on the caller side, since Do itself cannot be cancelled once it has joined an in-flight call.
In handler.go, the PriceHandler exposes the cache over HTTP. Each request extracts the symbol query parameter and calls cache.Get, so a burst of simultaneous requests for the same symbol results in a single downstream load. The handler surfaces backend errors as 502 and encodes the price as JSON, keeping the concurrency machinery entirely inside the cache layer where it belongs.
The cache_test.go tab proves the behavior: it launches many goroutines against a loader that increments an atomic counter and sleeps, then asserts the loader ran far fewer times than the number of callers. This is the canonical way to verify deduplication — count real executions, not returned values. The trade-offs are worth noting: singleflight only helps for concurrent identical keys, it does not bound total throughput, and a slow loader will make every collapsed caller wait as long as the slowest single execution. It is the right tool for read-heavy, high-fan-in keys where recomputation is expensive and duplication is wasteful.
Share this code
Here's the card — post it anywhere.
