如何使用Python结合Selenium获取Chrome DevTools Network栏数据？

Hey there! I’ve dealt with this exact need before—pulling data from Chrome DevTools’ Network tab (specifically the Name column) while using Selenium. Let me walk you through a solid, working solution step by step:

Selenium lets you interact with Chrome's DevTools Protocol (CDP), the same toolset that powers DevTools under the hood. We’ll enable Network monitoring via CDP, capture every request the browser sends, and extract the values that show up in the Name column.

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
import time
import os  # Optional, for trimming URLs to match DevTools' Name display

# Configure Chrome settings
chrome_options = Options()
chrome_options.add_argument("--start-maximized")  # Optional, but helpful for visibility

# Initialize the browser (replace the chromedriver path with your own)
service = Service("/path/to/your/chromedriver")
driver = webdriver.Chrome(service=service, options=chrome_options)

# Enable Network monitoring via CDP
driver.execute_cdp_cmd("Network.enable", {})

# List to store all entries from the Network tab's Name column
network_request_names = []

# Callback function to capture each request's "Name" value
def grab_request_name(event):
    request_data = event.get("request")
    if request_data:
        # By default, we'll use the full request URL
        request_name = request_data.get("url")
        
        # Uncomment below to match DevTools' default Name display (only the file/path end)
        # request_name = os.path.basename(request_data.get("url"))
        
        network_request_names.append(request_name)

# Attach the listener to capture requests as they're sent
driver.add_event_listener("Network.requestWillBeSent", grab_request_name)

# Navigate to your target page
driver.get("https://your-target-website.com")

# Wait for page to fully load (adjust time based on your site's speed)
time.sleep(4)

# Clean up: stop listening and disable Network monitoring
driver.remove_event_listener("Network.requestWillBeSent", grab_request_name)
driver.execute_cdp_cmd("Network.disable", {})

# Print out all captured Name entries
print("All entries from Network tab's Name column:")
for idx, name in enumerate(network_request_names, 1):
    print(f"{idx}. {name}")

# Close the browser
driver.quit()

• Network.enable is mandatory: You have to run this CDP command first—otherwise, no request data will be captured.
• Choose the right event: Network.requestWillBeSent captures every request the browser sends (including scripts, stylesheets, images, and API calls). If you only want requests that received a response, use Network.responseReceived instead.
• Match DevTools' Name display: The default Name column shows the end of the URL (e.g., script.js instead of https://site.com/js/script.js). Use os.path.basename() to replicate that behavior (uncomment the line in the code).
• Better waiting than sleep: For production code, replace time.sleep() with Selenium's explicit waits (WebDriverWait) to wait for specific elements or conditions—this makes your script more reliable.

If you need to capture requests that fire after an interaction (like clicking a button or submitting a form), adjust the code to trigger the action before waiting:

from selenium.webdriver.common.by import By

# After navigating to the page...
submit_button = driver.find_element(By.CSS_SELECTOR, "button.submit")
submit_button.click()

# Wait for the dynamic requests to complete
time.sleep(3)

# Then proceed to collect and print the request names as before

Just make sure your ChromeDriver version matches your installed Chrome browser version—mismatches will cause errors.

内容的提问来源于stack exchange，提问作者West