使用requests.get下载SEC form4.xml与Firefox结果不符问题
问题描述
我尝试下载SEC网站上的form4.xml文件,对应目录链接为https://www.sec.gov/Archives/edgar/data/1965040/000112760224010272/,使用的Python代码如下:
import requests headers = { 'User-Agent': 'sec_datascrape', 'From': 'myemail@gmail.com', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', } url = "https://www.sec.gov/Archives/edgar/data/1965040/000112760224010272/form4.xml" r = requests.get(filing_href, headers=headers) with open('data.xml', 'wb') as f: f.write(r.content)
但运行后获取的是HTML页面(片段如下),而非期望的XML内容(期望内容附后),请问问题出在哪里?
获取到的HTML片段:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <meta http-equiv="Last-Modified" content="Fri, 15 Mar 2024 20:49:57 GMT" /> <title>EDGAR Filing Documents for 0001127602-24-010272</title> <link rel="stylesheet" href="/edgar/search/global/css/bootstrap/bootstrap.min.css" type="text/css" /> <link rel="stylesheet" type="text/css" href="/include/interactive2.css" /> </head> <body style="margin: 0; font-size: 16px; "> <!-- SEC Web Analytics - For information please visit: https://www.sec.gov/privacy.htm#collectedinfo --> <noscript><iframe src="//www.googletagmanager.com/ns.html?id=GTM-TD3BKV" height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript> <script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= '//www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); })(window,document,'script','dataLayer','GTM-TD3BKV');</script> <!-- End SEC Web Analytics --> <noscript><div style="color:red; font-weight:bold; text-align:center;">This page uses Javascript. Your browser either doesn't support Javascript or you have it turned off. To see this page as it is meant to appear please use a Javascript enabled browser.</div></noscript> <!-- BEGIN BANNER --> <div id="header" style="text-align: center;"> <nav id="main-navbar" class="navbar navbar-expand"> <ul class="navbar-nav"> <li class="nav-item"> <a class="nav__sec_link" href="https://www.sec.gov"> <img src="/edgar/search/images/edgar-logo-2x.png" alt="" style="height:6.25rem"> </a> </li> <li class="nav-item"> <a class="nav__sec_link" href="https://www.sec.gov"> <span class="link-text d-inline">SEC.gov</span> </a> </li> <li class="nav-item"> <a class="nav__link" href="//www.sec.gov/submit-filings/about-edgar" id="edgar-short-form"><span class="link-text">EDGAR</span></a> </li> </ul> <ul class="navbar-nav ml-auto"> <li class="nav-item"> <a href="/cgi-bin/browse-edgar?action=getcurrent" class="nav__link">Latest Filings</a> </li> <li class="nav-item"> <a href="https://www.sec.gov/edgar/search-and-access" class="nav__link">Filings search tools</a> </li> </ul> </nav> <div style="position: absolute;width: 100%;"> <h1 style="position: relative;top: -60px;">Filing Detail</h1></div> </div> <!-- END BANNER --> <!-- BEGIN BREADCRUMBS --> <div id="breadCrumbs"> <ul> <li><a href="/index.htm">SEC Home</a> »</li> <li><a href="/edgar/searchedgar/companysearch.html">Company Search</a> »</li> <li class="last">Current Page</li> </ul> </div> <!-- END BREADCRUMBS -->
期望获取的XML内容:
<?xml version="1.0"?> <ownershipDocument> <schemaVersion>X0508</schemaVersion> <documentType>4</documentType> <periodOfReport>2024-03-13</periodOfReport> <issuer> <issuerCik>0001965040</issuerCik> <issuerName>Fortrea Holdings Inc.</issuerName> <issuerTradingSymbol>FTRE</issuerTradingSymbol> </issuer> <reportingOwner> <reportingOwnerId> <rptOwnerCik>0001983802</rptOwnerCik> <rptOwnerName>Warren Amanda M.</rptOwnerName> </reportingOwnerId> <reportingOwnerAddress> <rptOwnerStreet1>8 MOORE DRIVE</rptOwnerStreet1> <rptOwnerStreet2></rptOwnerStreet2> <rptOwnerCity>DURHAM</rptOwnerCity> <rptOwnerState>NC</rptOwnerState> <rptOwnerZipCode>27709</rptOwnerZipCode> <rptOwnerStateDescription></rptOwnerStateDescription> </reportingOwnerAddress> <reportingOwnerRelationship> <isOfficer>1</isOfficer> <officerTitle>Chief Accounting Officer</officerTitle> </reportingOwnerRelationship> </reportingOwner> <aff10b5One>0</aff10b5One> <derivativeTable> <derivativeTransaction> <securityTitle> <value>Restricted Stock Unit</value> </securityTitle> <conversionOrExercisePrice> <footnoteId id="F1"/> </conversionOrExercisePrice> <transactionDate> <value>2024-03-13</value> </transactionDate> <transactionCoding> <transactionFormType>4</transactionFormType> <transactionCode>A</transactionCode> <equitySwapInvolved>0</equitySwapInvolved> </transactionCoding> <transactionTimeliness></transactionTimeliness> <transactionAmounts> <transactionShares> <value>3856</value> </transactionShares> <transactionPricePerShare> <value>0</value> </transactionPricePerShare> <transactionAcquiredDisposedCode> <value>A</value> </transactionAcquiredDisposedCode> </transactionAmounts> <exerciseDate> <footnoteId id="F2"/> </exerciseDate> <expirationDate> <footnoteId id="F2"/> </expirationDate> <underlyingSecurity> <underlyingSecurityTitle> <value>Common Stock</value> </underlyingSecurityTitle> <underlyingSecurityShares> <value>3856</value> </underlyingSecurityShares> </underlyingSecurity> <postTransactionAmounts> <sharesOwnedFollowingTransaction> <value>18825</value> <footnoteId id="F3"/> </sharesOwnedFollowingTransaction> </postTransactionAmounts> <ownershipNature> <directOrIndirectOwnership> <value>D</value> </directOrIndirectOwnership> </ownershipNature> </derivativeTransaction> </derivativeTable> <footnotes> <footnote id="F1">Each Restricted Stock Unit ("RSU") represents the right to receive, at settlement, one share of Fortrea Holdings Inc. ("Fortrea") Common Stock. This transaction represents the settlement of RSUs in shares of Common Stock on their scheduled vesting date.</footnote> <footnote id="F2">The RSUs vest in three equal annual installments beginning on March 13, 2025.</footnote> <footnote id="F3">This number reflects the aggregate number of RSUs held by the reporting person.</footnote> </footnotes> <ownerSignature> <signatureName>/s/ James Stillman Hanson, Attorney-in-Fact for Amanda M. Warren</signatureName> <signatureDate>2024-03-15</signatureDate> </ownerSignature> </ownershipDocument>
问题原因及解决办法
变量名错误
你已经把目标XML的URL定义到url变量,但请求时调用的是未定义的filing_href变量。如果实际代码中filing_href指向的是目录页面(即不带form4.xml的链接),返回自然是目录的HTML页面,而非目标XML文件。修正后的代码
将请求时的变量替换为定义好的url,同时保留符合SEC爬虫规范的headers(你的headers已经包含必要的User-Agent和From字段,无需修改):
import requests headers = { 'User-Agent': 'sec_datascrape', 'From': 'myemail@gmail.com', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', } url = "https://www.sec.gov/Archives/edgar/data/1965040/000112760224010272/form4.xml" # 修正:使用定义好的url变量发起请求 r = requests.get(url, headers=headers) # 可选:打印状态码确认请求是否成功(200为成功) print(r.status_code) with open('data.xml', 'wb') as f: f.write(r.content)
- 额外验证提示
如果修正后仍有问题,可先手动访问该XML URL确认是否能正常获取内容。另外,SEC对爬虫请求频率有限制,避免短时间内大量请求导致访问被限制。
内容的提问来源于stack exchange,提问作者punyakoti
相关产品推荐
相关产品推荐

