Python requests Library Connection Pool Full Warning Under Heavy Load

The "Connection pool is full, discarding connection" warning from urllib3 (used internally by the requests library) means your application is making more simultaneous connections to a given host than the connection pool was configured to retain — the fix is either increasing the pool size to match your actual concurrency, or using a properly shared Session object in the first place.

The Problem

Under concurrent load — many simultaneous requests to the same host — the application logs a recurring warning:

WARNING:urllib3.connectionpool:Connection pool is full, discarding connection: api.example.com

The requests themselves might still succeed despite the warning, but performance suffers since connections are being discarded and new ones established repeatedly, rather than being efficiently reused as intended.

Why It Happens

requests (via urllib3) maintains a pool of reusable connections per host, with a default maximum pool size that's relatively conservative — sized for typical, moderate usage rather than high-concurrency workloads. When more simultaneous requests to the same host are in flight than the pool's configured capacity, excess connections are established but then discarded rather than retained for reuse, since the pool has no room to keep them for future reuse. Common causes:

• Making many concurrent requests to the same host (via threading, or otherwise), exceeding the default pool size (commonly 10) without adjusting the configuration to match your actual concurrency needs
• Creating a new requests.Session() (or worse, not using a Session at all, making raw requests.get() calls) for every individual request rather than reusing a single, shared Session configured with appropriately sized connection pooling
• Running many concurrent worker threads or async tasks that each independently make HTTP requests to the same external service, collectively exceeding a pool sized for much lower concurrency

The Fix

First, ensure you're using a shared Session object rather than making individual, unpooled requests — this is a prerequisite for connection pooling to provide any benefit at all, separate from the specific pool size question:

import requests

session = requests.Session()

# Reuse this same session across every request, rather than
# calling requests.get() directly each time, or creating a new
# Session() per request

Configure the connection pool size explicitly to match your actual expected concurrency, using an HTTPAdapter with appropriate pool settings:

from requests.adapters import HTTPAdapter

adapter = HTTPAdapter(pool_connections=50, pool_maxsize=50)
session.mount('https://', adapter)
session.mount('http://', adapter)

pool_maxsize specifically controls how many connections are retained per host for reuse — raising it to comfortably exceed your actual peak concurrent request count to any single host eliminates the warning, since the pool now has sufficient capacity to retain connections rather than discarding excess ones.

Choose a pool size based on your actual, measured concurrency rather than an arbitrarily large number — a reasonable starting point is slightly above your expected peak concurrent requests to a single host, adjusted based on observed behavior under real load:

# If you expect up to ~30 concurrent requests to the same host at peak,
# size the pool with some headroom above that
adapter = HTTPAdapter(pool_connections=40, pool_maxsize=40)

If you're making requests from multiple threads, ensure the shared Session object itself is thread-safe for this usage pattern — requests.Session is documented as thread-safe for making concurrent requests, so a single shared session with an appropriately sized pool, used across multiple threads, is the correct pattern rather than creating separate sessions per thread:

from concurrent.futures import ThreadPoolExecutor

session = requests.Session()
adapter = HTTPAdapter(pool_connections=50, pool_maxsize=50)
session.mount('https://', adapter)

def fetch(url):
    return session.get(url)

with ThreadPoolExecutor(max_workers=30) as executor:
    results = list(executor.map(fetch, urls))

Matching pool_maxsize to roughly your ThreadPoolExecutor's max_workers count (or somewhat higher, with headroom) is a sensible approach when the concurrency is bounded by an explicit worker pool like this, since you know the actual maximum possible simultaneous requests directly from that configuration.

Still Not Working?

If you've properly sized the connection pool and are using a shared session, but the warning persists, check whether requests are actually being made to many different hosts rather than one — connection pool sizing is per-host, so a warning appearing despite generous pool_maxsize configuration might indicate the concurrency is genuinely spread across many distinct hostnames, each with its own separate pool, rather than being concentrated on a single host where your specific size adjustment would directly apply. In that case, consider whether pool_connections (which controls the number of distinct host pools cached, separate from pool_maxsize's per-host connection limit) also needs adjustment to accommodate the actual number of distinct hosts being contacted.