为Python中Selenium多会话脚本添加代理支持的方法

Hey there! I’ve dealt with this exact proxy issue in Selenium before—let’s get your random proxies working for each browser session properly. Here’s what you need to know:

You’ll need a list of working proxies to pull from randomly. These can be free (note: often unreliable) or paid (much more trustworthy). Format each proxy as either http://ip:port or http://username:password@ip:port if authentication is required. Example:

PROXY_POOL = [
    "http://192.168.1.100:8080",
    "http://proxy-user:secure-pass@203.0.113.5:3128",
    "http://10.0.0.5:8888",
    # Add more valid proxies here
]

The most common reason your initial Proxy setup failed is that it wasn’t properly attached to the browser’s options. Below are reliable implementations for Chrome and Firefox, which are the most commonly used browsers with Selenium:

For Chrome Sessions #

Use ChromeOptions to directly inject the proxy into the browser launch arguments—this method is more consistent than the standalone Proxy class:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import random

def create_chrome_session(proxy):
    chrome_options = Options()
    # Inject the proxy into Chrome's launch arguments
    chrome_options.add_argument(f"--proxy-server={proxy}")
    
    # Optional: Add anti-bot detection settings to avoid bans
    chrome_options.add_argument("--disable-blink-features=AutomationControlled")
    chrome_options.add_experimental_option("excludeSwitches", ["enable-automation"])
    chrome_options.add_experimental_option("useAutomationExtension", False)
    
    # Launch the browser with the configured options
    driver = webdriver.Chrome(options=chrome_options)
    return driver

# Example: Create 3 unique sessions with random proxies
for _ in range(3):
    random_proxy = random.choice(PROXY_POOL)
    driver = create_chrome_session(random_proxy)
    
    # Test if the proxy is working (check your public IP)
    driver.get("https://api.ipify.org")
    print(f"Session IP: {driver.page_source}")
    
    # Navigate to your target URL and perform tasks
    driver.get("https://your-target-url.com")
    
    # Don't forget to clean up when done
    # driver.quit()

For Firefox Sessions #

Firefox uses a slightly different approach with the Proxy class attached to FirefoxOptions:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.proxy import Proxy, ProxyType
import random

def create_firefox_session(proxy):
    # Set up the proxy object
    proxy_config = Proxy()
    proxy_config.proxy_type = ProxyType.MANUAL
    proxy_config.http_proxy = proxy
    proxy_config.ssl_proxy = proxy  # Critical for HTTPS sites
    
    firefox_options = Options()
    firefox_options.proxy = proxy_config
    
    # Optional anti-bot settings
    firefox_options.set_preference("dom.webdriver.enabled", False)
    firefox_options.set_preference("useAutomationExtension", False)
    
    driver = webdriver.Firefox(options=firefox_options)
    return driver

# Example usage
for _ in range(3):
    random_proxy = random.choice(PROXY_POOL)
    driver = create_firefox_session(random_proxy)
    
    driver.get("https://api.ipify.org")
    print(f"Session IP: {driver.page_source}")
    driver.get("https://your-target-url.com")
    
    # driver.quit()

• You might have only configured the HTTP proxy but not the HTTPS proxy (most modern sites use HTTPS, so this is a must).
• The standalone Proxy class doesn’t always apply correctly unless explicitly attached to the browser’s options object.
• If your proxies require authentication, you forgot to include the username/password in the proxy string.

• Validate Proxies First: Use a quick requests check to remove dead proxies from your pool before running sessions.
• Rotate Responsibly: Don’t reuse the same proxy for too many sessions in a row, and avoid hitting the target site too frequently.
• Invest in Paid Proxies: Free proxies are often slow, already blocked, or monitored—paid residential proxies are far less likely to get flagged.
• Layer Anti-Detection: The anti-bot settings included in the code above help Selenium avoid being identified as an automated tool, which is a common reason for bans even with proxies.

内容的提问来源于stack exchange，提问作者Damdam31