How do you handle race conditions and use locks in threaded Python code?

Why the GIL doesn't prevent race conditions

import threading

counter = 0

def increment():
    global counter
    for _ in range(100_000):
        counter += 1   # NOT atomic: read counter, add 1, write back -- 3 separate steps

threads = [threading.Thread(target=increment) for _ in range(4)]
[t.start() for t in threads]
[t.join() for t in threads]
print(counter)   # often less than 400,000 -- lost updates!

counter += 1 compiles to multiple bytecode instructions (load, add, store). The GIL guarantees each individual bytecode instruction runs atomically, but a thread can be swapped out between those instructions. So two threads can both read the same value before either writes back the incremented result, and one increment gets lost.

Fixing it with a Lock

counter = 0
lock = threading.Lock()

def increment():
    global counter
    for _ in range(100_000):
        with lock:              # only one thread executes this block at a time
            counter += 1

threads = [threading.Thread(target=increment) for _ in range(4)]
[t.start() for t in threads]
[t.join() for t in threads]
print(counter)   # always exactly 400,000

with lock: acquires the lock on entry and releases it on exit (even if an exception occurs). While held, no other thread can enter a block guarded by the same lock, making the read-modify-write sequence atomic with respect to other threads using that lock.

Other synchronization primitives

• RLock (reentrant lock): the same thread can acquire it multiple times (recursive functions or nested methods that both need the lock, for example) without deadlocking itself. A plain Lock would deadlock in that case.
• Semaphore(n): allows up to n threads to hold it simultaneously. Useful for capping concurrent access to a limited resource, like at most 5 concurrent connections to a rate-limited service.
• Condition: lets threads wait for some condition to become true, signaled by another thread (.wait()/.notify()). This is the basis for producer/consumer patterns.
• Event: a simple flag threads can wait on until another thread sets it (.set()), useful for one-shot "start now" or "shutdown" signals.

Avoiding locks altogether: prefer thread-safe data structures/patterns

from queue import Queue

q = Queue()   # thread-safe by design -- internally uses its own locking

def producer():
    q.put("item")

def consumer():
    item = q.get()

queue.Queue is thread-safe internally, so a producer/consumer pattern built around it needs no manual locking at all. Preferring built-in thread-safe structures (Queue, queue.LifoQueue) over hand-rolled locking is usually safer and simpler than reasoning about locks directly.

The GIL makes individual bytecode instructions atomic, but not multi-instruction operations like x += 1. Race conditions are still possible and must be guarded with threading.Lock (or a higher-level primitive like Queue) around any critical section that reads and writes shared mutable state.