如何在Polars读取Minio（S3）Parquet文件时关闭SSL证书验证

问题场景 #

尝试从部署自签名证书的Minio（S3兼容）实例读取Parquet文件，使用代码如下：

import polars as pl
storage_options = {
"aws_access_key_id": "s3-key",
"aws_secret_access_key": "s3secretkey",
"endpoint_url": "https://<s3instance>.com",
}
file_path = 's3://<bucketname>/data.parquet'
pl.read_parquet(file_path, storage_options=storage_options)

执行后报错：

polars.exceptions.ComputeError: Generic S3 error: Error after 2 retries in 1.91494864s, max_retries:2, retry_timeout:10s, source:error sending request for url (https://<s3instance>/<bucketname>/data.parquet): error trying to connect: invalid peer certificate: UnknownIssuer

已尝试两种跳过证书验证的配置，但均无效：

• 配置client_kwargs：

storage_options = {
"aws_access_key_id": "s3-key",
"aws_secret_access_key": "s3secretkey",
"endpoint_url": "https://<s3instance>.com",
"client_kwargs": {"verify": "False"}
}

• 直接添加verify参数：

storage_options = {
"aws_access_key_id": "s3-key",
"aws_secret_access_key": "s3secretkey",
"endpoint_url": "https://<s3instance>.com",
"verify": "False"
}

解决方案 #

Polars底层依赖fsspec处理S3存储，s3fs实现要求跳过证书验证的参数需放在config_kwargs下，且verify必须是布尔类型而非字符串。正确配置如下：

import polars as pl
storage_options = {
    "aws_access_key_id": "s3-key",
    "aws_secret_access_key": "s3secretkey",
    "endpoint_url": "https://<s3instance>.com",
    "config_kwargs": {"verify": False}
}
file_path = 's3://<bucketname>/data.parquet'
pl.read_parquet(file_path, storage_options=storage_options)

关键注意点：之前尝试使用字符串"False"而非布尔值False，这是配置不生效的核心原因。

如果需要信任特定自签名证书，也可将verify设置为证书文件路径：

"config_kwargs": {"verify": "/path/to/your/certificate.pem"}

内容的提问来源于stack exchange，提问作者mr-price