Why Your Python Cache Does the Same Work Twice (P.S: GIL Won’t Save You)

[Sohel Alam]

You added a cache so an expensive function would run once per key. Then you counted how often it really ran. For 50 keys, it ran about 400 times. The cache works. Every value it returns is correct. It just doesn’t do the one job you gave it. And the usual excuse, “Python has a GIL, so threads can’t collide,” doesn’t apply here.



In my last article, two objects shared one list by accident. Here, threads share a cache on purpose and still get surprised. The bug is not in the shared data. It is in the gap between two lines of code.



I ran every example in this article on CPython 3.12.3, the standard build with the GIL. Exact numbers will differ on your machine. The pattern won’t.

The Bug: A Cache That Doesn’t Cache​

Here is a small cache and eight threads that all ask for the same 50 keys:

import threading
import time

calls = []      # records every time expensive() really runs
cache = {}

def expensive(key):
    calls.append(key)
    time.sleep(0.001)          # stands in for a database query or an HTTP call
    return key * 2

def get(key):
    if key not in cache:                # 1. check
        cache[key] = expensive(key)     # 2. act
    return cache[key]

def run_threads(target, n=8, args=()):
    threads = [threading.Thread(target=target, args=args) for _ in range(n)]
    for t in threads:
        t.start()
    for t in threads:
        t.join()

def worker(getter):
    for key in range(50):
        getter(key)

run_threads(worker, args=(get,))

print(len(cache))   # 50
print(len(calls))   # around 400, not 50 (changes from run to run)

Each key should be computed once. Instead, expensive() ran roughly 8 × 50 = 400 times, almost as if there were no cache at all. On some runs the number is a little lower, because a few threads arrive after a value has already been stored.

calls.append() is a single operation on a built-in list, so it is safe to use for counting here.

“But Python Has a GIL”​

In the standard CPython build, the GIL lets only one thread run Python bytecode at a time. It protects the interpreter’s own data structures from being corrupted when threads run together.



What it does not do is make your two-step logic atomic. Between the check (key not in cache) and the act (cache[key] = ...), the interpreter is free to switch to another thread. That switch happens when a thread waits on I/O or sleeps, and it can also happen on a timer. expensive() is exactly the kind of function that waits, so it opens a wide gap.



One line to remember: the GIL protects the interpreter, not your logic.

Step by Step: How Two Threads Collide​

Take two threads, A and B, that both ask for key 5 at the same moment:

Step	Thread A	Thread B	Cache state
1	Checks key 5: missing	—	Empty
2	Calls expensive(5) and sleeps	—	Empty
3	—	Checks key 5: missing	Empty
4	—	Calls expensive(5) and sleeps	Empty
5	Stores cache[5] = 10	—	{5: 10}
6	—	Stores cache[5] = 10	{5: 10}

Both threads did the full work, and the second write simply replaced the first. With 8 threads and 50 keys, the same collision repeats again and again. This pattern has a name: check-then-act. The decision (“it’s missing”) is made on information that is already stale by the time you act on it.

Why the Classic counter += 1 Demo Can Show Nothing​

Most thread-safety tutorials start with several threads doing counter += 1 and showing lost updates. I tried exactly that: 4 threads, 500,000 increments each, on Python 3.12.3. I got 2,000,000 every time, with no lost updates.



Two lessons come out of that. First, a clean result is not a guarantee. The language does not promise that a multi-step update is atomic, and what you see depends on the Python version and on timing. Second, a check-then-act gap that contains a sleep, a network call, or another function call is much wider than a gap of a few bytecodes, and the cache bug reproduced on every run I tried.

When Wasted Work Becomes a Wrong Answer​

Wasted work is the mild version. Here is the same race breaking a rule your code was supposed to enforce:

balance = 100
barrier = threading.Barrier(5)

def withdraw(amount):
    global balance
    if balance >= amount:           # check
        barrier.wait()              # demo only: all threads finish checking before anyone subtracts
        balance -= amount           # act

run_threads(withdraw, n=5, args=(100,))
print(balance)   # -400

Five threads each try to withdraw 100 from a balance of 100. The Barrier is a demo device: it holds every thread until all five have passed the check, so the race happens on every run. In real code, the gap is a fraud check, a database call, or any other point where a thread can wait, and whether the race shows up depends on timing, so it may appear only occasionally. Here, every thread passed the check before any thread subtracted, and the balance ends at -400.

Fix 1: One Lock Around Check and Act​

The fix is to make the check and the act one indivisible step, by holding the same lock for both:

lock = threading.Lock()

def get_safe(key):
    with lock:
        if key not in cache:
            cache[key] = expensive(key)
        return cache[key]

calls.clear()
cache.clear()
run_threads(worker, args=(get_safe,))

print(len(cache))   # 50
print(len(calls))   # 50

Now each key is computed exactly once. The balance example gets the same treatment:

balance = 100
balance_lock = threading.Lock()

def withdraw_safe(amount):
    global balance
    with balance_lock:
        if balance >= amount:       # check
            balance -= amount       # act

run_threads(withdraw_safe, n=5, args=(100,))
print(balance)   # 0

The trade-off: while one thread runs expensive() under the lock, threads that want other keys wait too. That is fine for small jobs. For slow work, you want something finer.

Fix 2: One Lock Per Key​

Here only threads asking for the same key wait for each other:

locks = {}
locks_guard = threading.Lock()

def get_per_key(key):
    if key in cache:                    # fast path: no lock for keys already cached
        return cache[key]
    with locks_guard:                   # short lock, only to fetch this key's lock
        key_lock = locks.setdefault(key, threading.Lock())
    with key_lock:                      # slow work: only threads asking for the SAME key wait
        if key not in cache:            # check again, now that we hold the lock
            cache[key] = expensive(key)
    return cache[key]

calls.clear()
cache.clear()
run_threads(worker, args=(get_per_key,))

print(len(cache))   # 50
print(len(calls))   # 50

The second if key not in cache inside the lock matters. A thread may have waited on the lock while another thread filled the cache, so it must check again before doing the work.



Three caveats. The fast path assumes entries are never removed or replaced, so if your cache evicts or updates entries, guard that read too. The locks dictionary grows with the number of keys, so this version suits a bounded set of keys. And threading.Lock only coordinates threads inside one process: it does nothing for two worker processes, two servers, or a restart.

The Surprising One: lru_cache Can Run Your Function More Than Once​

You might think functools.lru_cache already solves this. The documentation says the cache is thread-safe, which means its internal data stays consistent. It also says the wrapped function can be called more than once if another thread calls it before the first call has finished and been cached. Thread-safe here means “won’t corrupt itself,” not “will run your function once.”

from functools import lru_cache

calls.clear()

@lru_cache(maxsize=None)
def cached_expensive(key):
    calls.append(key)
    time.sleep(0.001)
    return key * 2

run_threads(worker, args=(cached_expensive,))
print(len(calls))   # around 400 again, not 50
print(cached_expensive.cache_info())
# CacheInfo(hits=0, misses=400, maxsize=None, currsize=50)   <- my run, yours will differ

The cache ends up holding the right 50 values, but in my run all 400 calls missed before any result was stored, so the expensive function ran 400 times. Your numbers will differ: a thread that arrives after a result has been cached gets a hit instead.



If you need to prevent duplicate concurrent computations for the same key within one process, put a per-key lock in front of it:

calls.clear()
key_locks = {}
key_locks_guard = threading.Lock()

@lru_cache(maxsize=None)
def _cached(key):
    calls.append(key)
    time.sleep(0.001)
    return key * 2

def cached_once(key):
    with key_locks_guard:
        key_lock = key_locks.setdefault(key, threading.Lock())
    with key_lock:
        return _cached(key)

run_threads(worker, args=(cached_once,))
print(len(calls))   # 50

A rule of thumb: if duplicate work is cheap and harmless (a pure function), plain lru_cache is fine. If the work is expensive, a per-key lock cuts duplicate work inside one process. It does not make anything exactly-once: exceptions, retries, cache eviction, and other processes can all still run the function again. For side effects such as payments, use idempotency keys and persistent deduplication. An in-process lock alone is not an exactly-once guarantee.



functools.cached_property has the same kind of caveat. The documentation warns that its getter could run more than once on the same instance. In my test on Python 3.12, eight threads reading it at the same time ran it eight times:

from functools import cached_property

calls.clear()

class Report:
    @cached_property
    def data(self):
        calls.append(1)
        time.sleep(0.05)
        return 42

report = Report()
run_threads(lambda: report.data)
print(len(calls))   # 8 on my run: once per thread

What Free-Threaded Python Changes (and What It Doesn’t)​

CPython also has a free-threaded build that runs without the GIL. Python 3.14 removed its “experimental” label: the Steering Council accepted PEP 779, which makes the free-threaded build officially supported but still optional. Whether it ever becomes the default is a separate, later decision.



The cache bug in this article does not need free-threading. It happens on the regular build with the GIL, as the runs above show. What free-threading takes away is the idea that the GIL’s scheduling can act as a safety net for your code. If your logic quietly depended on “it probably won’t switch here,” explicit synchronization, such as locks and queues, is what holds up on both builds.

I did not run these examples on a free-threaded build, so treat that last part as direction, not measurement.

A Practical Checklist​

• Search your code for the shape if x not in shared: shared[x] = ... and for if shared.value > 0: shared.value -= 1.
• For each one, ask: between the check and the act, can anything wait (I/O, sleep, another function call)? If yes, another thread can run in that gap.
• Put the check and the act under the same lock.
• Prefer sharing less: give each thread its own data, or pass results through a queue.Queue.
• Reproduce before you fix. Count how often the work really runs under many threads.

Key Takeaway: The GIL protects the interpreter’s internals, not your logic. If a decision depends on shared state, the check and the action must happen under the same lock.

Final Thoughts​

In my last article, the shared object was a list hiding in a class. In Why Your Python Functions Are Secretly Changing Data You Never Passed to Them, it was a list hiding in a default argument. Here, it is a dictionary that every thread trusts. It is the same lesson with higher stakes: shared mutable state is fine until two pieces of code look at it at different moments and both assume it hasn’t changed.

So the next time a cache, a counter, or a balance gives you a strange number, ask yourself: what happened between my check and my action?