如何配置RDFLib发起网络请求时显式使用指定代理
- 需使用RDFLib解析远程RDF文件,网络环境要求所有请求必须通过代理,不知道如何为RDFLib配置代理参数
- 初始调用代码:
import rdflib g = rdflib.Graph() g.parse(url)
- 解析地址
https://spec.edmcouncil.org/fibo/ontology/LOAN/LoansGeneral/Loans/Loan时抛出连接错误,错误栈如下:
Traceback (most recent call last): File "C:\Users\d72704\PycharmProjects\Ontology\main.py", line 113, in
Read_RDF() File "C:\Users\d72704\PycharmProjects\Ontology\main.py", line 21, in Read_RDF g.parse(r'https://spec.edmcouncil.org/fibo/ontology/LOAN/LoansGeneral/Loans/Loan') File "C:\Users\d72704\AppData\Roaming\Python\Python39\site-packages\rdflib\graph.py", line 1234, in parse source = create_input_source( File "C:\Users\d72704\AppData\Roaming\Python\Python39\site-packages\rdflib\parser.py", line 326, in create_input_source ) = _create_input_source_from_location( File "C:\Users\d72704\AppData\Roaming\Python\Python39\site-packages\rdflib\parser.py", line 375, in _create_input_source_from_location input_source = URLInputSource(absolute_location, format) File "C:\Users\d72704\AppData\Roaming\Python\Python39\site-packages\rdflib\parser.py", line 218, in init file = _urlopen(req) File "C:\Users\d72704\AppData\Roaming\Python\Python39\site-packages\rdflib\parser.py", line 206, in _urlopen return urlopen(req) File "C:\Program Files\Python39\lib\urllib\request.py", line 214, in urlopen return opener.open(url, data, timeout) File "C:\Program Files\Python39\lib\urllib\request.py", line 517, in open response = self._open(req, data) File "C:\Program Files\Python39\lib\urllib\request.py", line 534, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "C:\Program Files\Python39\lib\urllib\request.py", line 494, in _call_chain result = func(*args) File "C:\Program Files\Python39\lib\urllib\request.py", line 1389, in https_open return self.do_open(http.client.HTTPSConnection, req, File "C:\Program Files\Python39\lib\urllib\request.py", line 1349, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [WinError 10054] An existing connection was forcibly closed by the remote host>
- 从错误栈确认RDFLib底层调用
urllib.request发起请求,已验证requests库传入proxies参数可正常访问目标地址,可用代码片段如下:
proxies = {myproxyhere} r = requests.get(myurlhere, proxies=proxies, timeout=5)
- 存在权限限制:无Windows管理员权限无法修改系统代理,配置PyCharm内置代理时抛出
Problem with connection: Received fatal alert: protocol_version错误,代理配置无法生效。
RDFLib的parse方法不强制要求传入远程URL,支持直接传入已经获取到的文本/字节内容、本地文件对象进行解析,因此不需要强行修改RDFLib的内部请求逻辑,有两种可直接运行的方案:
方案1:用已调通的requests拉取内容后交给RDFLib解析(优先推荐)
这种方案完全不需要修改系统配置、不需要调整IDE代理设置,所有逻辑在代码内可控,同时requests的TLS兼容性更好,能避开遇到的protocol_version报错
import rdflib import requests # 替换为自己的实际代理配置 proxies = { "http": "http://代理地址:端口", "https": "http://代理地址:端口" } target_url = "https://spec.edmcouncil.org/fibo/ontology/LOAN/LoansGeneral/Loans/Loan" # 走代理拉取远程RDF内容 resp = requests.get(target_url, proxies=proxies, timeout=10) resp.raise_for_status() # 遇到4xx/5xx错误直接抛出,避免解析无效内容 g = rdflib.Graph() # 把拉取到的文本传入解析,FIBO本体默认是turtle格式,显式指定格式可避免自动检测出错 g.parse(data=resp.text, format="turtle") # 后续正常操作RDF图即可 print(f"解析完成,共加载{len(g)}条三元组")
方案2:代码内临时配置urllib代理(无额外依赖)
如果不想安装requests依赖,可以在代码运行时临时给urllib安装带代理的opener,配置仅在当前Python进程内生效,不会修改系统设置:
import rdflib from urllib import request # 配置代理处理器 proxy_handler = request.ProxyHandler({ "http": "http://代理地址:端口", "https": "http://代理地址:端口" }) opener = request.build_opener(proxy_handler) # 将自定义opener注册为urllib默认实现,RDFLib调用urlopen时会自动走代理 request.install_opener(opener) g = rdflib.Graph() g.parse("https://spec.edmcouncil.org/fibo/ontology/LOAN/LoansGeneral/Loans/Loan")
该方案如果遇到TLS版本报错,可以自行调整urllib的SSL上下文,优先使用方案1。
内容的提问来源于stack exchange,提问作者Connor

