多线程 + requests 并发请求

44.10 多线程 + requests 并发请求

多个 URL 同时请求时,I/O 等待期间线程可切换,总耗时接近最慢单个而非相加。

需 pip install requests;注意控制并发数,勿对同一站点造成过大压力。

更规范做法:ThreadPoolExecutor + submit(fetch, url)。

并发 GET 请求

# ========================================
# 示例:多线程 requests
# 需 pip install requests
# ========================================
import threading
import requests

urls = [
    'https://httpbin.org/delay/1',
    'https://httpbin.org/delay/1',
    'https://httpbin.org/get'
]

def fetch(url):
    r = requests.get(url, timeout=15)
    print(url[:40], '...', r.status_code)

threads = [threading.Thread(target=fetch, args=(u,)) for u in urls]
for t in threads:
    t.start()
for t in threads:
    t.join()
print('全部完成')