如何简化循环中请求头编写?每次请求需随机User Agent
问题描述
我想简化请求头的编写方式——现在每个循环里都要重复写完整的请求头字典,目的是每次请求都随机生成新的User Agent,但我不想在每个循环里都重复写这段代码。
原代码示例:
for example in examples: headers = { 'accept': '*/*', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6', 'referer': 'https://www.google.com/', 'user-agent': random.choice(all_user_agents), } response = request.get(url, headers=headers) while 10 > i: headers = { 'accept': '*/*', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6', 'referer': 'https://www.google.com/', 'user-agent': random.choice(all_user_agents), } response = request.get(url, headers=headers) for test in tests: headers = { 'accept': '*/*', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6', 'referer': 'https://www.google.com/', 'user-agent': random.choice(all_user_agents), } response = request.get(url, headers=headers)
解决方案
方法1:封装生成请求头的函数
把固定的请求头模板抽离出来,写一个函数每次返回带随机UA的完整请求头,避免重复编写字典结构:
import random # 定义固定的基础请求头(只写一次) BASE_HEADERS = { 'accept': '*/*', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6', 'referer': 'https://www.google.com/' } def get_random_headers(): # 复制基础头,防止修改原字典影响后续请求 headers = BASE_HEADERS.copy() headers['user-agent'] = random.choice(all_user_agents) return headers # 循环中直接调用函数获取请求头 for example in examples: response = request.get(url, headers=get_random_headers()) while 10 > i: response = request.get(url, headers=get_random_headers()) for test in tests: response = request.get(url, headers=get_random_headers())
方法2:复用基础字典动态添加UA
如果不想额外写函数,也可以在每次请求时复制基础字典,再添加随机UA,代码更轻量化:
BASE_HEADERS = { 'accept': '*/*', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6', 'referer': 'https://www.google.com/' } for example in examples: headers = BASE_HEADERS.copy() headers['user-agent'] = random.choice(all_user_agents) response = request.get(url, headers=headers) while 10 > i: headers = BASE_HEADERS.copy() headers['user-agent'] = random.choice(all_user_agents) response = request.get(url, headers=headers) for test in tests: headers = BASE_HEADERS.copy() headers['user-agent'] = random.choice(all_user_agents) response = request.get(url, headers=headers)
方法3:封装完整的请求方法
如果重复的不仅是请求头,连request.get的调用逻辑也一致,可以把整个请求逻辑封装成函数,进一步简化代码:
BASE_HEADERS = { 'accept': '*/*', 'accept-language': 'en-GB,en-US;q=0.9,en;q=0.8,es;q=0.7,ru;q=0.6', 'referer': 'https://www.google.com/' } def get_with_random_ua(url): headers = BASE_HEADERS.copy() headers['user-agent'] = random.choice(all_user_agents) return request.get(url, headers=headers) # 调用时只需传入URL即可 for example in examples: response = get_with_random_ua(url) while 10 > i: response = get_with_random_ua(url) for test in tests: response = get_with_random_ua(url)
内容的提问来源于stack exchange,提问作者BQuist
相关产品推荐
相关产品推荐

