Skip to content

About

Reproducible Python HTTP client experiments: httpx timeouts, aiohttp connection limits, 429 Retry-After vs token bucket, proxy passwords with special characters (Proxy302)

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

python-http-client-lab: timeouts, connection limits, 429s and proxy passwords, reproduced

Four small experiments that answer common "why does my Python HTTP client do that?" questions with numbers instead of guesses. Each script starts its own misbehaving test server on 127.0.0.1, so you can run them offline and see the same results: httpx's four timeouts (its default is 5 s, and read/write are gaps between bytes, not totals), aiohttp's default 100-connection limit and what limit=0 does, why retrying 429s on Retry-After alone causes a storm and how a token bucket fixes it, and what happens when a proxy password contains @ # : /. Proxy302 maintains this repository and sells proxies; nothing here depends on whose proxy you use.

Maintained by Proxy302. MIT licensed. Tested on macOS with Python 3.9.6, httpx 0.28.1, aiohttp 3.13.5, curl 8.7.1.

Run

pip install httpx aiohttp
python3 httpx_timeouts.py
python3 aio_limits.py
python3 ratelimit.py
python3 authproxy.py

1. httpx timeouts: httpx_timeouts.py

default timeout, silent server         ReadTimeout             5.09s
read=2, silent server                  ReadTimeout             2.01s
read=2, drip 1 byte/0.8s (20 bytes)    200 20 bytes           16.12s
connect=2, unroutable 10.255.255.1     ConnectTimeout          2.02s
write=2, 64MB upload, server not reading WriteTimeout            2.03s
write=None, read=3, same upload        ReadError              30.02s
max_connections=1, pool=1: req1: 200 2.04s; req2: PoolTimeout 1.00s
max_connections=1, pool=5: req1: 200 2.01s; req2: 200 4.01s
  • httpx times out after 5 s by default; requests has no default timeout.
  • read and write limit the gap between chunks, so a server that drips data never trips them. Enforce a total deadline yourself (stream and check the clock, or asyncio.wait_for).
  • Don't set write=None for uploads: a stalled peer holds the request until it closes the socket.
  • PoolTimeout means more concurrent requests than Limits(max_connections=...) allows, not a slow network.

2. aiohttp connection limits: aio_limits.py

default connector                     n=200 max_concurrent=100 tcp_conns=100 1.05s errors=0
limit=10                              n=50  max_concurrent=10  tcp_conns=10  2.52s errors=0
limit=0 (no limit)                    n=200 max_concurrent=128 tcp_conns=128 0.55s errors=72 ['ClientConnectorError']
limit_per_host=5, 2 hosts             n=40  max_concurrent=10  tcp_conns=10  3.03s errors=0
Semaphore(10), default connector      n=50  max_concurrent=10  tcp_conns=10  2.53s errors=0
force_close=True, limit=10            n=50  max_concurrent=10  tcp_conns=50  2.55s errors=0
  • TCPConnector defaults: limit=100, limit_per_host=0; default ClientTimeout(total=300, sock_connect=30) with no read timeout.
  • limit=0 overflowed the test server's listen backlog (aiohttp's TCPSite default is 128) and 72 of 200 requests failed to connect.
  • ClientTimeout.connect includes time spent waiting for a free connection in the pool.

3. 429 and Retry-After: ratelimit.py

The server allows 5 requests per 1-second window and answers 429 with Retry-After beyond that. 30 concurrent requests:

no limiter, give up on 429         n=30 ok=5   429=25   0.00s
no limiter, honor Retry-After      n=30 ok=30  429=75   5.04s
token bucket 5/s, capacity 5       n=30 ok=30  429=7    6.41s
token bucket 4.5/s, capacity 1     n=30 ok=30  429=0    6.49s
  • Retrying only after 429 works but wastes requests: every rejected client sleeps the same Retry-After and retries together. Add jitter.
  • A bucket at exactly the limit with a burst capacity still overshoots a fixed window. Run slightly under the limit with capacity 1.

4. Proxy passwords with special characters: authproxy.py

The local proxy requires Basic auth with password p@ss#w:rd/1:

encoded password: p%40ss%23w%3Ard%2F1
httpx, raw password in URL    -> ConnectError (name resolution)   proxy saw: no request
httpx, percent-encoded        -> 200   proxy saw user:pass = u-demo:p@ss#w:rd/1
aiohttp, raw password in URL  -> ClientConnectorDNSError: Cannot connect to host ss:80
aiohttp, percent-encoded      -> 200
aiohttp, proxy_auth=BasicAuth -> 200

curl behaves the same: raw password in -x exits with code 5 (could not resolve proxy); percent-encoded or -U 'user:password' works. The symptom of an unencoded password is a DNS error for a host you never typed, not a 407. Encode with urllib.parse.quote(pwd, safe="").

Related

Results recorded 21 November 2026.

About

Reproducible Python HTTP client experiments: httpx timeouts, aiohttp connection limits, 429 Retry-After vs token bucket, proxy passwords with special characters (Proxy302)

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages