Python批量下载PDF时遍历URL遇InvalidSchema报错求助

从网站收集唯一标识符生成400余个PDF文件的URL列表，编写for循环遍历下载时持续报错，相关代码和错误信息如下：

原代码 #

# Import van pakketten python
import os
import pandas as pd
import requests
from selenium import webdriver
from time import sleep

# Defineren van bestandslocaties
map_archive = r'C:\folder\\'
map_huidig_boekjaar = r'C:\folder\\'

# Download gemeentecodes
if not os.path.exists(r'C:\folder\Gemeenten alfabetisch 2022.xlsx'):
    url_gemeentecodes = 'https://www.cbs.nl/-/media/cbs/onze-diensten/methoden/classificaties/overig/gemeenten-alfabetisch-2022.xlsx'

    options = webdriver.ChromeOptions()
    prefs = {"download.default_directory": map_archive}
    options.add_experimental_option("prefs", prefs)

    driver = webdriver.Chrome(executable_path='C:\\chromedriver.exe', options=options)
    driver.get(url_gemeentecodes)
    sleep(2)
    driver.quit()

df = pd.read_excel(r'C:\folder\Gemeenten alfabetisch 2022.xlsx', dtype='str')
print(df.head())

# Defineren locatie bestanden huidig boekjaar
bron_url_1 = "https://findo.nl/jaarstukken/Jaarstukken_2018_GEMEENTE18_"
bron_url_2 = ".pdf"

Jaarrekening_url_2021 = bron_url_1 + df["Gemeentecode"].astype(str) + bron_url_2
Jaarrekening_url_2021.to_csv('url_list.txt', index=False, header=False, sep=",")
Bestand_naam = map_huidig_boekjaar + df["Gemeentecode"].astype(str) + bron_url_2

# Download bestanden 2021
def download_pdf(url, filename):

    # Send GET request
    response = requests.get(url)

    # Save the PDF
    if response.status_code == 200:
        with open(filename, "wb") as f:
            f.write(response.content)
    else:
        print(response.status_code, filename)

for url in Jaarrekening_url_2021 :
   download_pdf(Jaarrekening_url_2021, Jaarrekening_url_2021)

错误信息 #

Traceback (most recent call last):
  File "C:\Users\meind\SynologyDrive\6. Python projecten\SiSa\Jaarrekeningen gemeente downloaden.py", line 52, in <module>
    download_pdf(Jaarrekening_url_2021, Jaarrekening_url_2021)
  File "C:\Users\meind\SynologyDrive\6. Python projecten\SiSa\Jaarrekeningen gemeente downloaden.py", line 43, in download_pdf
    response = requests.get(url)
  File "C:\Users\meind\SynologyDrive\6. Python projecten\SiSa\venv\lib\site-packages\requests\api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "C:\Users\meind\SynologyDrive\6. Python projecten\SiSa\venv\lib\site-packages\requests\api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "C:\Users\meind\SynologyDrive\6. Python projecten\SiSa\venv\lib\site-packages\requests\sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\meind\SynologyDrive\6. Python projecten\SiSa\venv\lib\site-packages\requests\sessions.py", line 695, in send
    adapter = self.get_adapter(url=request.url)
  File "C:\Users\meind\SynologyDrive\6. Python projecten\SiSa\venv\lib\site-packages\requests\sessions.py", line 792, in get_adapter
    raise InvalidSchema(f"No connection adapters were found for {url!r}")
requests.exceptions.InvalidSchema: No connection adapters were found for '0      https://findo.nl/jaarstukken/Jaarstukken_2018_...\n1      https://findo.nl/jaarstukken/Jaarstukken_2018_...\n2      https://findo.nl/jaarstukken/Jaarstukken_2018_...\n3      https://findo.nl/jaarstukken/Jaarstukken_2018_...\n4      https://findo.nl/jaarstukken/Jaarstukken_2018_...\n                             ...                        \n340    https://findo.nl/jaarstukken/Jaarstukken_2018_...\n341    https://findo.nl/jaarstukken/Jaarstukken_2018_...\n342    https://findo.nl/jaarstukken/Jaarstukken_2018_...\n343    https://findo.nl/jaarstukken/Jaarstukken_2018_...\n344    https://findo.nl/jaarstukken/Jaarstukken_2018_...\nName: Gemeentecode, Length: 345, dtype: object'

Process finished with exit code 1

错误根源 #

错误的核心是for循环中的函数传参逻辑完全错误：

• Jaarrekening_url_2021是pandas的Series对象，存储了所有生成的PDF URL。你写的循环for url in Jaarrekening_url_2021本意是遍历每个单个URL，但调用download_pdf时，却把整个Series对象同时传给了url和filename参数。
• 这导致requests.get()收到的不是合法的单个URL字符串，而是整个Series的文本表示（包含索引、换行、省略号的长串），requests无法识别这种格式的"URL"，因此抛出InvalidSchema错误。

修正方案 #

只需要修改for循环部分，同时遍历URL序列和对应的文件名序列，将单个的url和filename传入下载函数：

# 替换原错误循环
for url, filename in zip(Jaarrekening_url_2021, Bestand_naam):
    download_pdf(url, filename)

额外注意事项 #

• 代码中bron_url_1包含年份2018，但变量名是Jaarrekening_url_2021，明显是年份笔误，建议检查并修正，确保URL指向正确年份的文件。
• 可以在download_pdf函数末尾添加sleep(1)（确保已导入time模块），避免短时间内发起大量请求被目标网站限制访问。

内容的提问来源于stack exchange，提问作者Meindert Wenting