使用Scrapy FormRequest.from_response登录西门子ASP.NET站点失败

我尝试登录https://signin.siemens.com/regpublic/login.aspx以查看关联域名的物料库存。站点接收POST请求后，未完成登录流程就跳转到https://www.automation.siemens.com/（预期应显示403错误，说明已在其他端登录）。我的表单请求负载和手动登录的正确请求完全匹配，但登录失败。

已尝试的操作：

• 添加clickdata（包含点击/不点击场景）、指定formname，均无效
• Cookie收集正常
• 确认__EVENTTARGET字段为空，排除该字段问题
• 完全匹配手动成功请求的格式，但不确定是否正确定位表单

当前Scrapy登录代码 #

def start_requests(self):
    login_url = "https://signin.siemens.com/regpublic/login.aspx"
    yield scrapy.Request(login_url, callback=self.login)

def login(self, response):
    logging.debug("starting login")
    req = FormRequest.from_response(
        response,
        method="POST",
        formdata={
            "ctl00$GlobalScriptManager": "ctl00$ContentPlaceHolder1$LoginUserNamePasswordUpdatePanel|ctl00$ContentPlaceHolder1$LoginUserNamePasswordButton",
            "__LASTFOCUS": "",
            "GlobalScriptManager_TSM": "",
            "__EVENTTARGET": "",
            "__EVENTARGUMENT": "",
            "__VIEWSTATE": response.css(
                "input#__VIEWSTATE::attr(value)"
            ).extract_first(),
            "__VIEWSTATEGENERATOR": response.css(
                "input#__VIEWSTATEGENERATOR::attr(value)"
            ).extract_first(),
            "__EVENTVALIDATION": response.css(
                "input#__EVENTVALIDATION::attr(value)"
            ).extract_first(),
            "ctl00$ContentPlaceHolder1$TextSiemensLogin": "EMAIL_USERNAME_HERE",
            "ctl00$ContentPlaceHolder1$TextPassword": "PASSWORD_HERE", 
            "ctl00$ContentPlaceHolder1$hdflLoginUserNamePassword": "1",
            "ctl00$ContentPlaceHolder1$LoginUserNamePasswordButton": "Login",
            "__ASYNCPOST": "true",
        },
        callback=self.start_scraping,
    )
    logging.debug(req.headers)
    logging.debug(req.body)
    return req

手动成功请求格式（禁用JavaScript环境） #

__LASTFOCUS: 
GlobalScriptManager_TSM: 
__EVENTTARGET: 
__EVENTARGUMENT: 
__VIEWSTATE: [VIEW-STATE-EXTRACT GOES HERE]
__VIEWSTATEGENERATOR: 93C9AE53
__EVENTVALIDATION: [EVENT-VALIDATION-EXTRACT GOES HERE]
ctl00$ContentPlaceHolder1$TextSiemensLogin: [EMAIL_USERNAME_HERE]
ctl00$ContentPlaceHolder1$TextPassword: [PASSWORD_HERE]
ctl00$ContentPlaceHolder1$LoginUserNamePasswordButton: Login
ctl00$ContentPlaceHolder1$hdflLoginUserNamePassword: 

关键差异与当前考虑 #

手动请求中没有__ASYNCPOST: "true"，但Scrapy必须添加该参数才能完成请求。目前考虑改用Scrapy Splash模拟手动操作，希望在这之前获得解决方案建议。

内容的提问来源于stack exchange，提问作者cforcomputer