如何绕过urllib2.URLError?实现Python脚本自动重试请求
Handling Transient Network Errors in Mechanize Scripts
Great question—this URLError [Errno 0] is almost certainly a transient network issue (like a dropped connection, temporary DNS glitch, or server-side blip) that clears up on its own. The good news is you can absolutely add retry logic to your script to automatically recover without needing manual restarts. Here are two clean approaches:
1. Simple Retry Loop (Quick Implementation)
Wrap your br1.open() call in a try-except block with a retry counter. This is straightforward for one-off use cases:
import time from urllib2 import URLError def free(): max_retries = 5 # Adjust based on how many times you want to retry retry_delay = 3 # Seconds to wait between retries url = "your-target-url-here" # Replace with your actual URL for attempt in range(max_retries): try: srch = br1.open(url) # Proceed with your normal logic now that the request succeeded return srch except URLError as e: print(f"Attempt {attempt + 1} failed: {str(e)}. Retrying in {retry_delay}s...") time.sleep(retry_delay) # If all retries fail, either raise an error or handle it gracefully raise RuntimeError(f"Failed to connect to {url} after {max_retries} attempts. Please check your network or the target server.")
2. Reusable Retry Decorator (Cleaner for Multiple Functions)
If you have multiple functions making network calls, a decorator lets you reuse the retry logic without duplicating code:
import time from urllib2 import URLError from functools import wraps def retry_on_network_error(max_retries=5, delay=3): """Decorator to retry functions that throw URLError""" def decorator(func): @wraps(func) def wrapper(*args, **kwargs): for attempt in range(max_retries): try: return func(*args, **kwargs) except URLError as e: print(f"{func.__name__} attempt {attempt + 1} failed: {str(e)}. Retrying in {delay}s...") time.sleep(delay) raise RuntimeError(f"{func.__name__} failed after {max_retries} retries.") return wrapper return decorator # Apply the decorator to your free function @retry_on_network_error(max_retries=5, delay=3) def free(): srch = br1.open("your-target-url-here") # Your normal logic here return srch
Key Notes for Better Reliability
- Avoid infinite retries: Always set a
max_retrieslimit to prevent your script from getting stuck in a loop if the issue is permanent (e.g., the server is down). - Exponential backoff: For production use, consider increasing the delay between retries exponentially (e.g., 3s → 6s → 12s → 24s) to be kinder to the target server and give more time for network issues to resolve.
- Log instead of print: Replace
print()statements with Python'sloggingmodule to keep track of retries in a log file, which helps with debugging later. - Filter specific errors: If you want to only retry certain URLError subcases (e.g., connection timeouts), you can add checks inside the except block to skip retries for non-transient errors.
内容的提问来源于stack exchange,提问作者a.lejeune626




