为了防止过快的访问影响网站,爬虫需要模拟人类的行为,例如设置请求头、延迟请求、代理 IP 等<br>
headers = {<br> 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'<br>}<br>response = requests.get(url, headers=headers)<br>