Custom TTL + LRU Cache Decorator for Expensive Python Computations
import functools
import threading
import time
from collections import OrderedDict
def _make_key(args, kwargs):
if kwargs:
return args + tuple(sorted(kwargs.items()))
return args
def ttl_lru_cache(maxsize=128, ttl=60.0):
def decorator(func):
store = OrderedDict()
lock = threading.Lock()
stats = {"hits": 0, "misses": 0}
@functools.wraps(func)
def wrapper(*args, **kwargs):
key = _make_key(args, kwargs)
now = time.monotonic()
with lock:
entry = store.get(key)
if entry is not None and entry[1] > now:
store.move_to_end(key)
stats["hits"] += 1
return entry[0]
stats["misses"] += 1
value = func(*args, **kwargs)
with lock:
store[key] = (value, time.monotonic() + ttl)
store.move_to_end(key)
while len(store) > maxsize:
store.popitem(last=False)
return value
def cache_clear():
with lock:
store.clear()
stats["hits"] = 0
stats["misses"] = 0
def cache_info():
with lock:
return dict(stats, size=len(store), maxsize=maxsize, ttl=ttl)
wrapper.cache_clear = cache_clear
wrapper.cache_info = cache_info
return wrapper
return decorator
import time
from ttl_cache import ttl_lru_cache
@ttl_lru_cache(maxsize=256, ttl=2.0)
def compute_shipping_quote(origin, dest, weight_kg):
# Simulates a slow rate-table lookup / external call.
time.sleep(0.5)
base = abs(hash((origin, dest))) % 40 + 5
return round(base + weight_kg * 1.75, 2)
def main():
start = time.monotonic()
print(compute_shipping_quote("SFO", "JFK", 3.2))
print("cold call took %.2fs" % (time.monotonic() - start))
start = time.monotonic()
print(compute_shipping_quote("SFO", "JFK", 3.2))
print("warm call took %.3fs" % (time.monotonic() - start))
print(compute_shipping_quote.cache_info())
time.sleep(2.1) # let the TTL lapse
start = time.monotonic()
print(compute_shipping_quote("SFO", "JFK", 3.2))
print("expired call took %.2fs" % (time.monotonic() - start))
print(compute_shipping_quote.cache_info())
if __name__ == "__main__":
main()
This snippet builds a caching decorator that combines the eviction behavior of an LRU (least-recently-used) cache with per-entry TTL (time-to-live) expiry, then applies it to a genuinely expensive computation. The standard library's functools.lru_cache handles size-bounded eviction well, but it has no concept of staleness: once a value is memoized it lives until it is pushed out by capacity. Many real workloads — pricing tables, remote config, aggregate rollups — need the opposite guarantee, where a value is trusted only for a bounded window. The ttl_lru_cache decorator fills that gap without pulling in an external cache server.
In ttl_lru_cache decorator, the cache is a plain OrderedDict mapping a _make_key tuple to a (value, expires_at) pair. The OrderedDict is what makes the LRU part cheap: move_to_end promotes a key on every hit, and popitem(last=False) evicts the oldest entry when the store grows past maxsize. Freshness is enforced lazily inside wrapper: on each call the code looks up the key, and if the stored expires_at is in the past it treats the entry as a miss and recomputes. This lazy expiry avoids a background sweeper thread and means expired entries cost nothing until they are next requested.
Correctness under concurrency is the subtle part. A threading.Lock guards every mutation of the shared OrderedDict, because dictionaries are not safe against interleaved reads and structural writes across threads. Note that the expensive function itself runs while the lock is released — the lock is taken only to read the cached slot and again to store the result. This keeps the critical section short but does allow two threads to compute the same key concurrently on a cold miss; that trade-off favors throughput over strict single-flight execution, which is the right default for pure, idempotent functions. _make_key builds a hashable key from positional args plus sorted keyword items so that f(1, x=2) and f(1) with a later x=2 map distinctly.
The decorator also attaches a cache_clear and a cache_info helper to the wrapped function, mirroring the ergonomics of functools.lru_cache so callers can inspect hit and miss counters or reset state in tests. A monotonic time.monotonic clock is used for expiry rather than wall-clock time.time, so the TTL is immune to system clock adjustments.
In pricing.py, the decorator wraps compute_shipping_quote, a function that simulates a slow rate-table lookup. The main block demonstrates the observable behavior: repeated calls within the TTL return instantly from cache, a call after the window forces recomputation, and cache_info() reports the accumulated statistics. The pattern is worth reaching for whenever a function is pure, expensive, and its inputs repeat, but the underlying data drifts slowly enough that a few seconds of staleness is acceptable. Pitfalls to keep in mind: unhashable arguments will raise at _make_key, extremely high cardinality keys defeat the maxsize bound, and functions with side effects should never be memoized this way.
Share this code
Here's the card — post it anywhere.
