如何用Python的requests模块获取ICSI会员查询页面指定编号的会员姓名？

Your issue stems from missing critical dynamic form fields required by the ASP.NET backend, plus some unnecessary cookie handling. Here's how to get it working:

Why Your Original Code Failed #

ASP.NET forms rely on hidden anti-forgery tokens like __VIEWSTATE and __EVENTVALIDATION that are generated when you first load the page. Your code didn't retrieve these tokens, so the server rejected your POST request. Additionally, your cookie handling was redundant (requests.Session automatically persists cookies) and had a syntax error.

Corrected Approach #

1. First load the page to capture all required hidden fields.
2. Include all dynamic tokens in your POST data.
3. Use the same session to maintain cookies between requests.
4. (Optional) Add a valid User-Agent to avoid being flagged as a bot.

Working Code #

import requests
from lxml import html

# Initialize session and set a user-agent to mimic a browser
session = requests.Session()
session.headers.update({
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36'
})

url = "https://www.icsi.in/student/Members/MemberSearch.aspx"

# Step 1: GET the initial page to extract hidden fields
response = session.get(url)
tree = html.fromstring(response.content)

# Extract mandatory ASP.NET hidden fields
viewstate = tree.xpath('//input[@id="__VIEWSTATE"]/@value')[0]
viewstate_generator = tree.xpath('//input[@id="__VIEWSTATEGENERATOR"]/@value')[0]
event_validation = tree.xpath('//input[@id="__EVENTVALIDATION"]/@value')[0]

# Prepare POST data with all required fields
post_data = {
    '__VIEWSTATE': viewstate,
    '__VIEWSTATEGENERATOR': viewstate_generator,
    '__EVENTVALIDATION': event_validation,
    '__EVENTTARGET': 'dnn$ctr410$MemberSearch$btnSearch',
    '__EVENTARGUMENT': '',
    'dnn$ctr410$MemberSearch$txtCpNumber': '16803',
    'dnn$ctlHeader$dnnSearch$Search': 'SiteRadioButton'  # Keep this as per your original data
}

# Step 2: Send POST request to submit the search
response = session.post(url, data=post_data)
tree = html.fromstring(response.content)

# Extract the member name
name = tree.xpath('//div[@class="name_head"]//text()')
print("Member Name:", ''.join(name).strip())

Key Notes #

• Dynamic Fields: Always extract __VIEWSTATE, __VIEWSTATEGENERATOR, and __EVENTVALIDATION from the initial GET response—these values change with each page load.
• Session Persistence: Using requests.Session() ensures cookies (like session IDs) are retained between your GET and POST requests, which is required for the server to recognize your session.
• User-Agent: Adding a browser-like user-agent helps avoid being blocked by anti-scraping measures.

If you still run into issues, check if the form field IDs have changed (sometimes ASP.NET modifies control IDs based on page structure). You can inspect the page source to verify the exact IDs of the hidden fields and form inputs.

内容的提问来源于stack exchange，提问作者nikhil ss