Token bucket rate limiter for outbound calls

12032 views

                  package deps

import (
  "context"
  "net/http"

  "golang.org/x/time/rate"
)

type LimitedClient struct {
  client  *http.Client
  limiter *rate.Limiter
}

func NewLimitedClient(client *http.Client, r rate.Limit, burst int) *LimitedClient {
  return &LimitedClient{client: client, limiter: rate.NewLimiter(r, burst)}
}

func (c *LimitedClient) Do(ctx context.Context, req *http.Request) (*http.Response, error) {
  if err := c.limiter.Wait(ctx); err != nil {
    return nil, err
  }
  return c.client.Do(req.WithContext(ctx))
}

Outbound rate limiting is one of those “quiet reliability” features: customers rarely notice it until it’s missing. I prefer a simple token bucket using golang.org/x/time/rate because it’s well-tested and easy to reason about. Each call waits for a token via limiter.Wait(ctx); if the request is canceled or the deadline is exceeded, the wait returns an error and we skip the downstream call. This makes throttling cooperative and consistent with request timeouts. I usually attach a limiter per dependency, not per handler, so one noisy endpoint doesn’t starve everything else. The pattern also makes it easy to tune in production by adjusting rate.Limit and burst size based on real metrics.

Leah Thompson

More from Leah Thompson

Token bucket rate limiter for outbound calls

0 Comments

More from Leah Thompson