Python Requests库安全协议及内容提取安全性技术问询

Great question! Let's break this down clearly to cover all your concerns about the requests library's security behavior:

Under the hood, requests relies on the urllib3 library for low-level HTTP/HTTPS operations, so it inherits all of urllib3's secure protocol support:

• TLS/SSL Support: By default, it uses modern TLS versions (1.2 and newer, depending on your system's OpenSSL setup) and avoids outdated, insecure protocols like SSLv3 or TLS 1.0/1.1.
• Certificate Validation: SSL certificate verification is enabled by default. If you request a site with an invalid, self-signed, or expired certificate, requests will throw an SSLError to prevent man-in-the-middle attacks. You can disable this (not recommended!) with verify=False, but that's a deliberate choice on your part.
• Other Security Features: It supports certificate pinning, HTTP/2 (when available), and follows best practices for secure HTTP headers (like avoiding insecure redirects by default).

Here's a key point: requests does not perform automatic string escaping on response content. Its job is purely to fetch the raw HTTP response (whether that's HTML, JSON, JavaScript, or binary data) and pass it to you as-is.

String escaping is a concern when you process that content (e.g., inserting it into a database, rendering it in a web page, or executing it). For example:

• If you take raw HTML from a response and inject it into a web app without escaping, you could expose yourself to XSS attacks—but that's a mistake in your application logic, not a flaw in requests.
• For handling untrusted HTML safely, use libraries like BeautifulSoup which don't execute code, or use html.escape() if you need to display the content as plain text.

Let's cut to the chase: When you run r = requests.get('https://somesite.com'), nothing dangerous happens automatically.

requests is a pure HTTP client—it has no JavaScript engine. All it does is:

1. Send an HTTP GET request to the specified URL.
2. Receive the response headers and body (which may include the suspicious JS code as plain text).
3. Store that body in r.text (or r.content for binary data).

The JavaScript code is just a string in your Python program. It won't execute unless you explicitly use a separate library (like execjs, PyV8, or a headless browser such as Playwright) to run it. Even then, you'd be making a deliberate choice to execute untrusted code—requests doesn't trigger that on its own.

Quick Safety Reminders #

• Always leave certificate verification enabled unless you have a very good reason to disable it.
• Never execute raw, untrusted code (JS or otherwise) fetched from unknown sites.
• When processing response content, handle escaping based on the context (e.g., use html.escape() for HTML contexts, parameterized queries for databases).

内容的提问来源于stack exchange，提问作者Neel