使用代理请求URL时urllib2遇302重定向不允许错误的技术求助

Hey there, I’ve run into this exact issue before—urllib2’s default behavior doesn’t play nice with non-HTTP/HTTPS redirects like market:// (Android’s Google Play link protocol). Let’s break down why this happens and how to fix it.

Why You’re Seeing This Error #

The default HTTPRedirectHandler in urllib2 only permits redirects to URLs using http, https, or ftp protocols. When the server sends a 302 redirect to a market:// URL, urllib2 blocks it because this protocol isn’t in its allowed list, hence the "Redirection to url [...] is not allowed" message.

Solution 1: Custom Redirect Handler to Allow market:// #

You can create a custom redirect handler that explicitly allows market:// redirects. Here’s how to modify your code:

First, define the custom handler:

import urllib2

class MarketRedirectHandler(urllib2.HTTPRedirectHandler):
    def redirect_request(self, req, fp, code, msg, headers, new_url):
        # Allow redirects to market:// URLs
        if new_url.startswith('market://'):
            return urllib2.Request(new_url, headers=req.headers)
        # Fall back to default behavior for other protocols
        return super(MarketRedirectHandler, self).redirect_request(req, fp, code, msg, headers, new_url)

Then update your request setup to use this handler:

# Assume your proxy_handler is already defined (e.g., ProxyHandler({'http': 'proxy:port'}))
user_agent = "Your User Agent String"

# Build opener with proxy and custom redirect handler
opener = urllib2.build_opener(proxy_handler, MarketRedirectHandler())
opener.addheaders = [('User-Agent', user_agent)]
urllib2.install_opener(opener)

print('Performing request')
try:
    response = urllib2.urlopen(url)
    # If the redirect is allowed, you'll get a response for the market:// URL
    # (Note: urllib2 can't actually resolve market://, but this avoids the error)
except urllib2.HTTPError as e:
    print(f"Error: {e}")

Solution 2: Disable Auto-Redirect and Handle Manually #

If you don’t need to follow the market:// redirect (just want to capture the URL), you can disable auto-redirects and extract the Location header yourself:

import urllib2

proxy_handler = urllib2.ProxyHandler({'http': 'your_proxy:port', 'https': 'your_proxy:port'})
user_agent = "Your User Agent String"

# Build opener with proxy and disable auto-redirect
opener = urllib2.build_opener(proxy_handler)
opener.set_handle_redirect(False)  # Turn off automatic redirect handling
opener.addheaders = [('User-Agent', user_agent)]
urllib2.install_opener(opener)

print('Performing request')
try:
    response = urllib2.urlopen(url)
except urllib2.HTTPError as e:
    if e.code == 302:
        # Extract the redirect URL from the Location header
        redirect_url = e.headers.get('Location')
        print(f"Redirected to: {redirect_url}")
        # Here you can save this URL, process it, or use it as needed
    else:
        # Handle other HTTP errors
        print(f"HTTP Error: {e}")

Key Notes #

• market:// is a protocol specific to Android devices—urllib2 can’t actually "load" this URL like it does with HTTP, so your use case will dictate whether you need to just capture the URL or handle it differently.
• Make sure your proxy is configured correctly to pass through the redirect headers without blocking them.

内容的提问来源于stack exchange，提问作者Kishan Mehta