如何借助不同IP访问网站，验证其地域差异化内容？

Absolutely—using httr with proxies is exactly the right approach to test if website content changes based on location. Here’s a step-by-step guide tailored to your needs:

Step 1: Install & Load Required Packages #

First, make sure you have the tools you need. If you haven’t already, install and load httr (for handling HTTP requests with proxies) and rvest (for scraping content):

install.packages(c("httr", "rvest"))
library(httr)
library(rvest)

Step 2: Grab Working Proxy Details #

From the free proxy list you mentioned, pick a proxy that’s marked as "anonymous" or "elite" (these are less likely to be blocked). Note down its IP address and port—for example, let’s use a dummy proxy: 192.168.1.100 on port 8080.

Step 3: Configure the Proxy in Your Request #

httr has a built-in use_proxy() function to route your requests through a proxy. You can use this for single requests or set up a persistent session for multiple page loads.

Single Request Example #

# Define your proxy settings
proxy_ip <- "192.168.1.100"
proxy_port <- 8080

# Send a request to your target site via the proxy
proxy_response <- GET(
  url = "https://your-target-site.com",
  use_proxy(url = proxy_ip, port = proxy_port)
)

# Check if the request succeeded (status code 200 means all good)
status_code(proxy_response)

Persistent Session Example (Good for Navigating Multiple Pages) #

If you need to simulate a full browsing session through the proxy (like clicking links), create a persistent session:

# Set up a session with the proxy
proxy_session <- session(
  url = "https://your-target-site.com",
  config = use_proxy(url = proxy_ip, port = proxy_port)
)

# Navigate to a subpage and pull content
subpage_content <- proxy_session %>% 
  session_jump_to("/some-subpage") %>% 
  read_html()

Step 4: Scrape & Compare Content #

Once you have the response, use rvest to extract content and compare it with a request from your local IP:

# Extract page title via proxy
proxy_title <- proxy_response %>% 
  read_html() %>% 
  html_element("title") %>% 
  html_text()
cat("Title via proxy:", proxy_title, "\n")

# Extract page title via your local IP
local_response <- GET("https://your-target-site.com")
local_title <- local_response %>% 
  read_html() %>% 
  html_element("title") %>% 
  html_text()
cat("Title via local IP:", local_title, "\n")

Important Tips & Troubleshooting #

• Free Proxies Are Hit-or-Miss: Most free proxies are slow, get blocked quickly, or don’t work at all. You’ll likely need to test a few before finding one that works.
• Validate Your Proxy First: Before testing your target site, confirm the proxy is masking your IP. Use a simple IP-check service:

  ip_check <- GET("https://api.ipify.org", use_proxy(proxy_ip, proxy_port))
  cat("Proxy IP:", content(ip_check, "text"), "\n")
  

• Handle Errors Gracefully: Wrap your requests in tryCatch() to avoid crashes when a proxy fails:

  tryCatch({
    response <- GET("https://your-target-site.com", use_proxy(proxy_ip, proxy_port))
    # Process content here
  }, error = function(e) {
    cat("Proxy failed:", e$message, "\n")
  })
  

• Anonymity Matters: Some sites detect non-elite proxies and still show content based on your real location. Stick to "elite" proxies if you need full location spoofing.

Bonus: Test Multiple Proxies At Once #

If you want to test several proxies from your list, loop through them:

# Example list of proxies (replace with your own)
proxy_list <- data.frame(
  ip = c("192.168.1.100", "10.0.0.5"),
  port = c(8080, 3128)
)

for (i in 1:nrow(proxy_list)) {
  current_ip <- proxy_list$ip[i]
  current_port <- proxy_list$port[i]
  
  cat("\nTesting proxy:", current_ip, ":", current_port, "\n")
  
  tryCatch({
    # Check if proxy works
    ip_response <- GET("https://api.ipify.org", use_proxy(current_ip, current_port))
    cat("Working IP:", content(ip_response, "text"), "\n")
    
    # Fetch target site content
    target_response <- GET("https://your-target-site.com", use_proxy(current_ip, current_port))
    # Add your content comparison logic here
  }, error = function(e) {
    cat("Failed:", e$message, "\n")
  })
}

That’s all you need to start testing location-based content changes with R!

内容的提问来源于stack exchange，提问作者Francesco Dal Pont