functools
In short: a cache is a dict in a closure — remember each answer the first
time, hand back the remembered answer after. functools.lru_cache is that idea,
finished: a size bound with least-recently-used eviction, hit/miss stats via
.cache_info(), and .cache_clear(). A function is only safe to cache when its
result depends on nothing but its arguments — a global it reads can go
stale in the cache without anyone noticing, and the arguments themselves must be
hashable. functools.partial pre-fills some arguments into a new callable — a
closure you don't have to write by hand — and functools.reduce folds a
sequence into one value by repeated application.
Cache a slow host-to-region lookup
Resolving a host to a region means asking a slow directory — worth remembering once two million log rows only mention thirty distinct hosts:
The key is args, the tuple of arguments — it works because a tuple of strings
is hashable. Hold on to that; it comes back below.
Bound the region cache with lru_cache
lru_cache is a decorator factory — the last lesson's shape — where maxsize
is the setting. Over the hand-written version it adds a bound (past
maxsize, the least recently used entry is evicted to make room, the "LRU"),
statistics (cache_info()), and control (cache_clear()).
from functools import lru_cache
@lru_cache(maxsize=None)
def region_of(host):
return host[:2]
for host in ["eu-1", "ap-1", "eu-1", "eu-2", "eu-1", "ap-1"]:
region_of(host)
info = region_of.cache_info()
print(info.hits, info.misses)A cache smaller than the set of keys cycling through it can be worse than no cache at all:
from functools import lru_cache
calls = []
@lru_cache(maxsize=2)
def lookup(key):
calls.append(key)
return key
for key in ["a", "b", "c", "a", "b"]:
lookup(key)
print(len(calls))Watching a bounded cache fill and evict
With maxsize=2, only two answers fit. Step through the calls and watch which
key gets thrown out. It's the one used longest ago, not the one added
earliest:
Here's the decision the wrapper makes on each call:
That thrashing example above is exactly what this diagram predicts: each key arrives right after its own entry was evicted, so every call takes the miss branch.
Cache a tax total safely
The cache key has to be hashable, because it's a dict key:
TypeErrorRun it and read the error. Change the call so the argument can be used as a cache key.
The more serious rule is about meaning: a cache is only correct when the result depends on nothing but the arguments.
from functools import lru_cache
rate = 10
@lru_cache(maxsize=None)
def with_tax(price):
return price + price * rate // 100
first = with_tax(200)
rate = 20
print(first, with_tax(200))Before reaching for @lru_cache, check: the function is pure (no global,
file, clock or database read), its arguments are hashable, the result is
safe to share (a cached list is the same list handed to every caller — one
mutating it affects the rest), and memory is bounded — set a real maxsize
unless the key space is small and known.
One-liner: partial and reduce
partial pre-fills some arguments and hands back a new callable — a closure
you don't have to write yourself:
reduce folds a sequence into one value by applying a two-argument function
repeatedly — useful for composing a pipeline of steps, though a plain sum()
or "".join() reads clearer for those specific jobs:
A cache's hit rate climbs toward 100% only once lookups start repeating —
100 * (x - k) / x, for x total lookups and k distinct keys held entirely
in the cache. Few distinct keys and many repeats is exactly where a cache earns
its memory; many distinct keys with few repeats is the thrashing case above.
Quick practice
from functools import partial
def power(base, exp):
return base ** exp
square = partial(power, exp=2)
cube = partial(power, exp=3)
print(square(5), cube(2))from functools import lru_cache
@lru_cache(maxsize=None)
def slow(n):
return n * n
slow(2); slow(2); slow(3)
before = slow.cache_info().hits
slow.cache_clear()
after = slow.cache_info().hits
print(before, after)🎯 Your turn
Write cached_lookup(keys, maxsize). Inside it, decorate the provided
resolve(key) with a fresh lru_cache(maxsize=maxsize), count how many
times the real resolve body runs, look up every key in order, and return:
{"work": <real calls>, "hits": <cache hits>, "misses": <cache misses>}
cached_lookup(["a", "b", "a", "a"], 2)→{"work": 2, "hits": 2, "misses": 2}cached_lookup(["a", "b", "c", "a"], 2)→{"work": 4, "hits": 0, "misses": 4}(the"a"was evicted)
maxsize may be None, meaning unbounded.
Hint: count the real work with a small closure around resolve (a
nonlocal counter, as in lesson 01), then
cache = lru_cache(maxsize=maxsize)(counting_resolve) makes a fresh cache
every run. Read hits and misses from cache.cache_info().
Press ▶ Run, then ✓ Check. ✅