使用Python urllib提交请求无法修改网页input值的问题求助

Hey there! Let's break down why your current code isn't updating the input field and how to fix it.

Why Your Current Approach Isn't Working #

There are a couple of key issues with your code:

• Wrong Target URL: The form containing the wavelength input likely submits to a specific endpoint (not the root / URL you're using). You need to find the actual action attribute of the parent <form> element on the page.
• Incorrect Submit Parameter: The wavelength_submit button's value is ↵, not true—you need to match the exact value the form sends when submitted.
• Server-Side Rendering Dependency: Even if you send the right parameters, the input field will only show your value if the server renders the page with the submitted value in the input's value attribute. If the page uses JavaScript to populate values after load, a raw HTTP request won't make the input "appear" filled (since you're just getting the page source, not interacting with a browser).

Step-by-Step Fix #

1. Find the Form's Actual Submission Endpoint #

Open the refractiveindex.info page in your browser, right-click the wavelength input, and select "Inspect". Look for the parent <form> tag—check its action attribute. For example, it might be something like /database or a full URL.

2. Adjust Your Request Parameters and URL #

Update your code to use the correct endpoint and matching parameters. Here's a revised example (replace the url with the actual form action you found):

import urllib.parse
import urllib.request
import re

# Replace with the actual form action URL from the page
url = 'https://refractiveindex.info/database'
axicon_material = 'your_material_here'  # Fill in your actual material
used_wavelength = 'your_wavelength_here'  # Fill in your actual wavelength

# Use the exact parameter values the form expects
values = {
    'shelf': 'glass',
    'book': axicon_material,
    'wavelength': used_wavelength,
    'wavelength_submit': '↵'  # Match the button's value exactly
}

data = urllib.parse.urlencode(values).encode('utf-8')
index_mainpage_request = urllib.request.Request(url, data=data, method='POST')

with urllib.request.urlopen(index_mainpage_request) as response:
    page_source = response.read().decode('utf-8')
    # Check if the input has your value (look for value="your_wavelength")
    if f'value="{used_wavelength}"' in page_source:
        print("Wavelength parameter was successfully applied!")
    paragraphs = re.findall(r'<p>(.*?)</p>', page_source)

3. Verify with Browser DevTools #

To be 100% sure you're sending the right request:

• Open Chrome/Firefox DevTools (F12)
• Go to the "Network" tab
• Submit the form manually on the page
• Look at the request that gets sent (it'll be a POST request)
• Copy the URL, method, and all form parameters to match exactly in your code

If It Still Doesn't Work #

If the page uses JavaScript to handle the input value (instead of server-side rendering), you'll need to use a browser automation tool like Selenium to interact with the page directly. Here's a quick snippet for that:

from selenium import webdriver
from selenium.webdriver.common.by import By

driver = webdriver.Chrome()
driver.get('https://refractiveindex.info')

# Find the wavelength input and set its value
wavelength_input = driver.find_element(By.ID, 'wavelength')
wavelength_input.clear()
wavelength_input.send_keys(used_wavelength)

# Click the submit button
submit_button = driver.find_element(By.ID, 'wavelength_submit')
submit_button.click()

# Now you can extract the page content after interaction
page_source = driver.page_source
driver.quit()

内容的提问来源于stack exchange，提问作者user8584384