如何用Python从短链接列表获取最终URL?(ow.ly遇错误10061)
解决ow.ly短链接解析超时问题的几种方法
针对你遇到的ow.ly链接超时(TCP 10061错误)问题,下面是几个可行的解决思路:
1. 改用GET请求替代HEAD请求
很多短链接服务(包括ow.ly)会拒绝HEAD方法的请求,改用GET并开启stream=True可以在不下载完整响应内容的前提下获取跳转后的URL,效率也不会太低:
import requests results = [] num = 0 for item in list_of_urls: num += 1 try: # 改用GET + stream=True,避免下载大文件 r = requests.get(item, allow_redirects=True, timeout=5, stream=True) x = r.url print(f"{num}: {item} -> {x}") results.append(x) # 关闭连接,释放资源 r.close() except Exception as e: print(f"{num}: {item} 失败 - {str(e)}") continue
2. 添加浏览器请求头,模拟真实访问
ow.ly可能会检测请求的User-Agent,拒绝非浏览器的请求。添加常见的浏览器头信息可以绕过这种检测:
import requests # 模拟Chrome浏览器的请求头 headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36' } results = [] num = 0 for item in list_of_urls: num += 1 try: r = requests.get(item, allow_redirects=True, timeout=5, stream=True, headers=headers) x = r.url print(f"{num}: {item} -> {x}") results.append(x) r.close() except Exception as e: print(f"{num}: {item} 失败 - {str(e)}") continue
3. 添加重试机制,处理临时网络波动
TCP 10061错误有时候是临时的连接问题,通过重试适配器可以自动重试失败的请求:
import requests from requests.adapters import HTTPAdapter from urllib3.util.retry import Retry # 配置重试策略 retry_strategy = Retry( total=3, # 总重试次数 backoff_factor=1, # 重试间隔时间(1s, 2s, 4s...) status_forcelist=[429, 500, 502, 503, 504] # 需要重试的状态码 ) adapter = HTTPAdapter(max_retries=retry_strategy) session = requests.Session() session.mount("https://", adapter) session.mount("http://", adapter) # 加上浏览器头 headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36' } results = [] num = 0 for item in list_of_urls: num += 1 try: r = session.get(item, allow_redirects=True, timeout=5, stream=True, headers=headers) x = r.url print(f"{num}: {item} -> {x}") results.append(x) r.close() except Exception as e: print(f"{num}: {item} 失败 - {str(e)}") continue
4. 单独解析ow.ly链接(备选方案)
如果上面的方法都不行,可以尝试直接调用ow.ly的官方接口来解析短链接:
import requests headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/114.0.0.0 Safari/537.36' } def expand_owly(url): # 提取ow.ly的短码,比如从https://ow.ly/abc123提取abc123 short_code = url.split('/')[-1] api_url = f"https://ow.ly/api/1.1/link/expand?shortUrl=https://ow.ly/{short_code}" try: r = requests.get(api_url, headers=headers, timeout=5) data = r.json() if data.get('results'): return data['results'][0]['longUrl'] return url except: return url # 循环中判断是否是ow.ly链接,单独处理 results = [] num = 0 for item in list_of_urls: num += 1 try: if 'ow.ly' in item: x = expand_owly(item) else: r = requests.get(item, allow_redirects=True, timeout=5, stream=True, headers=headers) x = r.url r.close() print(f"{num}: {item} -> {x}") results.append(x) except Exception as e: print(f"{num}: {item} 失败 - {str(e)}") continue
内容的提问来源于stack exchange,提问作者travelsandbooks
相关产品推荐
相关产品推荐

